References

References

Here you can find optimizer code docs.

Scheduled Optim Pretraining

ScheduledOptimPretraining optimizer.

Scheduled Optim Finetuning

ScheduledOptimFinetuning optimizer