Stars
🧑🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), ga…
The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”