noahdgoodman
@noahdgoodman
Professor of natural and artificial intelligence @Stanford. Alignment at @GoogleDeepMind. (@StanfordNLP @StanfordAILab etc)
So proud! Go work with Gabriel, he’ll be the best advisor
Thrilled to join the UMich faculty in 2026! I'll also be recruiting PhD students this upcoming cycle. If you're interested in AI and formal reasoning, consider applying!
🔥 New position piece! 🔥 In this paper we lay out our vision for AI Alignment as guided by "Resource Rational Contractualism" (RRC). But wait -- what's that? A 🧵.
It’s like chain-of-thought for humans!
Can we record and study human chains of thought? The think-aloud method, where participants voice their thoughts as they solve a task, offers a way! In our #CogSci2025 paper co-led with Ben Prystawski, we introduce a method to automate analysis of human reasoning traces! (1/8)🧵
It turns out that a lot of the most interesting behavior of LLMs can be explained without knowing anything about architecture or learning algorithms. Here we predict the rise (and fall) of in-context learning using hierarchical Bayesian methods.
🚨New paper! We know models learn distinct in-context learning strategies, but *why*? Why generalize instead of memorize to lower loss? And why is generalization transient? Our work explains this & *predicts Transformer behavior throughout training* without its weights! 🧵 1/
congrats dr poesía!
Congratulations to @GabrielPoesia on receiving his @Stanford PhD today!
Every frontier AI system should be grounded in a core commitment: to protect human joy and endeavour. Today, we launch @LawZero_, a nonprofit dedicated to advancing safe-by-design AI. lawzero.org
Just released GDM’s 100+ page approach to AGI safety & security! (Don’t worry, there’s a 10 page summary.) AGI will be transformative. It enables massive benefits, but could also pose risks. Responsible development means proactively preparing for severe harms before they arise.
AGI could revolutionize many fields - from healthcare to education - but it's crucial that it’s developed responsibly. Today, we’re sharing how we’re thinking about safety and security on the path to AGI. → goo.gle/3R08XcD
🤔🤖Most AI systems assume there’s just one right answer—but many tasks have reasonable disagreement. How can we better model human variation? 🌍✨ We propose modeling at the individual-level using open-ended, textual value profiles! 🗣️📝 arxiv.org/abs/2503.15484 (1/?)
AI models are fascinating, impressive, and sometimes problematic. But what can they tell us about the human mind? In a new review paper, @noahdgoodman and I discuss how modern AI can be used for cognitive modeling: osf.io/preprints/psya…
“Four habits of highly effective STaRs” — we show that certain high level cognitive behaviors are necessary for learning to reason through RL. Exciting!
New Paper!! We try to understand why some LMs self-improve their reasoning while others hit a wall. The key? Cognitive behaviors! Read our paper on how the right cognitive behaviors can make all the difference in a model's ability to improve with RL! 🧵1/13
We raised a $22M Series A and are launching Elicit Reports, a better version of Deep Research for actual researchers. Elicit Reports are available for everyone to try right now, for free. 👇
Congrats to OAI on producing a reasoning model! Their opaque tweets demonstrate that they’ve (independently) found some of the core ideas that we did on our way to STaR.
Congrats to DeepSeek on producing an o1-level reasoning model! Their research paper demonstrates that they’ve independently found some of the core ideas that we did on our way to o1.
Presenting this tomorrow at @NeurIPSConf East Exhibit Hall A-C #2111 (4:30 p.m. PST — 7:30 p.m. PST). Come along if you want to chat about synthetic preference data with @gandhikanishk
Constitutional AI showed LMs can learn to follow constitutions by labeling their own outputs. But why can't we just tell a base model the principles of desired behavior and rely on it to act appropriately? Introducing SAMI: Self-Supervised Alignment with Mutual Information!
If you're at NeurIPS, come tomorrow for the Oral+Poster on "Learning Formal Mathematics from Intrinsic Motivation"! Really fun work with @DavidKarlBroman @nickhaber @noahdgoodman that put together much of what I did in the past years, w/ a new twist with open-ended learning!
Excited that @GabrielPoesia will be presenting his Oral on Learning Formal Mathematics From Intrinsic Motivation. We make and prove conjectures from scratch, without any human data, by learning what is hard but provable. Gabe’s on the job market, btw. neurips.cc/virtual/2024/o…