Zachary Nado
@zacharynado
Research eng @GoogleDeepMind on Gemini pretrain. Personal acct. Past: swe intern @SpaceX, ugrad researcher in @tserre lab @BrownUniversity. All opinions my own.
"Non-diagonal preconditioning has dethroned Nesterov Adam" 🧴👑 shampoo wins, finally the community can know what we have for years! this benchmark has been 3+ years in the making (we first talked about it Google in 2021), I'm beyond psyched that it's finally yielded results!
@MLCommons #AlgoPerf results are in! 🏁 $50K prize competition yielded 28% faster neural net training with non-diagonal preconditioning beating Nesterov Adam. New SOTA for hyperparameter-free algorithms too! Full details in our blog. mlcommons.org/2024/08/mlc-al… #AIOptimization #AI
Just imagine if they used Shampoo or Soap..
Anyone knows adam?
🧴🧴🧴
A Saturday reminder to all new followers that Shampoo stands for a preconditioner. It’s called Shampoo because thats what comes pre/before using a conditioner.
Google just discovered a powerful emergent capability in Veo 3 - visually annotate your instructions on the start frame, and Veo just does it for you! Instead of iterating endlessly on the perfect prompt, defining complex spatial relationships in words, you can just draw it out…
A deal with China along these lines would be an unmitigated disaster. In exchange for allowing more CCP penetration of the American homeland and U.S. companies, Beijing would get…critical technology? If China proposes this, the admin should laugh them out of the room.
Half a year into his second term, Trump’s China policy appears adrift. Now he is increasingly focused on trying to strike an economic bargain with Beijing A big question is, will the U.S. further relax export controls to get China to “open up?” wsj.com/economy/trade/… via @WSJ
Easily one of the funniest things i have seen in a long time. Perfect. No notes. Put it in a museum.
I am getting tired from influencers with little to no tech industry experience to write stupid stuff like this for likes. As @simonw said: “Quitting programming as a career right now because of LLMs would be like quitting carpentry thanks to the invention of the table saw.”
its over for cs grads. if you are in cs just leave the field. its never been so over. just pivot. go do something else but cs. do it now. its getting out of hand. anything but cs.
PSA: if you need anything (rate limits, questions, new features, etc) to scale up your usage of Gemini, Veo 3, Imagen, etc, please email me: [email protected] Here to help : )
I think the most important thing MAHA has done is on the salience side, yes. People are talking about this stuff. I'm talking about it. It's a bit akin to the way that Trump's policies get more ppl to talk about trade and trade balances. It's a floor raiser on salience for sure.…
🚢🚢🚢
Google is processing 980 trillion+ monthly tokens across our products and APIs (up from 480T in May) 🤯 No slowdown in sight, intelligence is everywhere.
the plan is this: you get a Coursera machine learning certificate. that alone will not get you a job at big tech but it will get you a guest lecture at MIT and with it an MIT email address. then you put MIT on all your social media. then start a tech podcast. then you wait.
I guess this is just life now
Megan Byron, wife of the guy who got caught bringing his not nearly as pretty side piece to a Coldplay concert, has issued a statement.
Gemini 2.5 Flash-Lite is now GA — it’s our fastest (400 tokens/second), most cost-efficient ($0.10 in, $0.40 out) 2.5 model yet. Look for it in Google AI Studio + Vertex AI. 🔦😉
kimi k2 (left) vs qwen3 coder (right)! prompt "the solar system scaled to fit inside a minecraft world"
> low value work (like data cleaning) uh...
Academia must be the only industry where extremely high-skilled PhD students spend much of their time doing low value work (like data cleaning). A 1st year management consultant outsources this immediately. Imagine the productivity gains if PhDs could focus on thinking