alex peysakhovich 🤖
@alex_peys
partner sutter hill ventures. ex-facebook ai. interests: dogs, ml, rl, game theory, multimodality, motorsports, embeddings, psych, bio, graphs
i used to say “the model was optimized with adam” to refer to @adamlerer
Anyone knows adam?
gpus going camping
We're rapidly expanding our AI infrastructure and have adopted a novel approach of building weather-proof tents to house GPU clusters. This enables us to get new data centers online in months instead of years. 🚀 Read more in this @FastCompany article: fastcompany.com/91369896/meta-…
🎶when you cast that last layer to fp32, that's numerics🎶
🎶when your grad becomes NaN and you have no idea why, that’s numerics🎶
🎶when your grad becomes NaN and you have no idea why, that’s numerics🎶
I am getting tired from influencers with little to no tech industry experience to write stupid stuff like this for likes. As @simonw said: “Quitting programming as a career right now because of LLMs would be like quitting carpentry thanks to the invention of the table saw.”
its over for cs grads. if you are in cs just leave the field. its never been so over. just pivot. go do something else but cs. do it now. its getting out of hand. anything but cs.
roberta is great, if you care about this open endedness (and you should care!) apply for her team
I’m building a new team at @GoogleDeepMind to work on Open-Ended Discovery! We’re looking for strong Research Scientists and Research Engineers to help us push the frontier of autonomously discovering novel artifacts such as new knowledge, capabilities, or algorithms, in an…
trump ai plan focuses really heavily on ai for science (and biomedical in particular), this is very good
It’s time for the American AI community to wake up, drop the "open is not safe" bullshit, and return to its roots: open science and open-source AI, powered by an unmatched community of frontier labs, big tech, startups, universities, and non‑profits. If we don’t, we’ll be forced…
Love to see this from @WhiteHouse!
i just wanna multiply matrices man, why do i have to worry about stuff like what optimizer i used in pretraining? god is cruel
There's evidence that Muon doesn't work as well for SFTing models not pretrained with Muon. This does seem like a huge limitation but hopefully Muon becomes popular enough for pretraining that it doesn't become an issue lol
people say dogs look/act like their owners, well, the same appears to be true of language models
Fascinating to compare the solutions of OpenAI vs Deepmind to the IMO 2025. Both won Gold for answering P1 to P5 correctly. OpenAI (left) vs Gemini (right)
People in the US are super dumb. You have a meal with some sort of Southerner who works in DC—it’s very depressing. They’re ignorant, they have terrible taste, they understanding nothing outside of the US, they know something about college football at best. And if you lived in a…
Thinking about this observation from @tylercowen again:
born to do linear algebra, forced to learn about bf16 vs fp32 numerics
having been burned by this a few times already in my life, im cautious, but... maybe RL is gonna work this time?
dario: we should have machines of loving grace grok: hold my beer
Grok just released a waifu AI companion that’s completely unfiltered. Grok 4 goon mode has been enabled.
the production function for basically any big researchy project is min(org quality, money, research scientist creativity, engineer autism) if you lack any of these you’re ngmi, a good leading indicator is # of meetings, if that is high something is wrong
It’s funny that people on this site think major LLM efforts are talent-bound rather than org-bound. The talent differential has never been big between major orgs. Most of the difference in outcomes is due to organisational factors - like allocating compute to the right bets, and…
I just saw @_albertgu call the major AI labs as "Big Token" and it has to be the most hilarious shit ever lol