Stephen Roller
@stephenroller
MoTS @thinkymachines. previously pre-training @googledeepmind ,@character_ai, and @aiatmeta.
I started Thinking Machines Lab alongside a remarkable team of scientists, engineers, and builders. We're building three things: - Helping people adapt AI systems to work for their specific needs - Developing strong foundations to build more capable AI systems - Fostering a…
We are moving incredibly fast. Come light up GPUs with us.
Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're…
Millennials use “lol” like STOP at the end of a telegram lol
Based on current administration policies, China will have an influx of returning talent and an accelerated advantage in research investment. You need to be both sinophobic and irrational to expect the US to continue as the global scientific powerhouse with these policies.
The White House Vision for Dismantling Science in One Simple Plot open.substack.com/pub/joshuaswei…
Revoking visas of Chinese students studying in critical fields like AI and Robotics is incredibly short-sighted and harmful to America’s long term prosperity. We want the best from every country to work for team America
The U.S. will begin revoking visas of Chinese students, including those with connections to the Chinese Communist Party or studying in critical fields.
The war on science in the US is already affecting private sector research like AlphaFold. Bears repeating but the private sector builds on top of things created by academic research for the public good. This hurts everyone.
American funding for hard sciences has fallen 2/3 this year. In physics, they are receiving 15% of what they did last year. What the fuck are we doing?
I once trained hyperbolic (Poincaré) networks with Riemannian SGD and HogWild. Your optimization stack does not scare me.
Not sure why the gutting of American science funding isn’t a bigger story. No one voted for it, it reduces American innovation and economic competitiveness in the near-term and long-term, and it isn’t even being done efficiently, if that were in fact the goal.
Computers used to scream out in pain when we connected to the internet. This was a warning and we did not heed it.
Today, we are excited to announce Thinking Machines Lab (thinkingmachines.ai), an artificial intelligence research and product company. We are scientists, engineers, and builders behind some of the most widely used AI products and libraries, including ChatGPT,…
float16 is just spicy integers pretending to be real numbers and i'm tired of pretending it's not
Our latest studies on the decoding text from brain activity, reviewed by MIT Technology Review @techreview: technologyreview.com/2025/02/07/111… Research done at @AIatMeta and @bcbl_ - Blog: ai.meta.com/blog/brain-ai-… - Study 1: ai.meta.com/research/publi… - Study 2: ai.meta.com/research/publi…
For friends of open source: imo the highest leverage thing you can do is help construct a high diversity of RL environments that help elicit LLM cognitive strategies. To build a gym of sorts. This is a highly parallelizable task, which favors a large community of collaborators.
So DeepSeek situation summarized: *They are not a small engineer team but one of the leading frontier lab (+100 researchers full time). *They are not a newcomer. Started in 2023 by retraining a llama, then slowly rising to the top. All documented in their 16 (!) papers.