Closed
Description
The instructlab userbase will have hardware that require various training configuration options
e.g. to train on colab I needed to disable bf16 and set
gradient_checkpointing=true
gradient_accumulation_steps=8
we should add a hidden, (perhaps temporary) argument to allow users to try various training options without
having to change the code
this could eventual be replaced with a happy set of arguments we know would cover all the cases
or code to auto sense the HW and set options based on what it finds.
Metadata
Metadata
Assignees
Labels
No labels