I stream daily on twitch: twitch.tv/xrsrke
Github: https://github.com/xrsrke
Feel free to get in touch at [email protected], @xariusrke on twitter, or neuralink on discord
Database
ZeRO-1: Distributed Optimizer
Non blocking all-reduce chimera
DiLoCo