K
Katie Everett
@_katieeverett
Machine learning researcher @GoogleDeepMind + PhD student @MIT. Opinions are my own.
Joined August 2013
632Following
3KFollowers
Katie Everett Retweeted
D
Damien Ferbach@damien_ferbach · May 26
It's very difficult to improve the *exponent* in scaling laws for loss vs compute, especially by changing the optimizer! Our new paper shows that scaling momentum correctly can *provably* improve the scaling exponent on a theoretical model. Empirically, it works on LSTMs too!
11
61
311
267
52.0K