Aditi Raghunathan
@AdtRaghunathan
Assistant professor at CMU @SCSatCMU @CSDatCMU | Machine learning
We got so tired of the cat 🐱 and mouse 🐭 game in LLM unlearning. What if we just made unlearning possible *by design*? This #ICML2025 work is a cool amalgamation of various findings around LLM training dynamics and memorization, all while keeping an eye towards scalability in…
1/So much of privacy research is designing post-hoc methods to make models mem. free. It’s time we turn that around with architectural changes. Excited to add Memorization Sinks to the transformer architecture this #ICML2025 to isolate memorization during LLM training🧵
If you are attending #ICML2025, check out our DataWorld workshop on Sat July 19. We have updated the website with more info on speakers & accepted papers! dataworldicml2025.github.io Also happy to chat offline about all things ✨ data ✨
We make a very intriguing observation that more pre-training data can worsen downstream performance. Come visit our poster at #ICML2025 on Thursday at East Exhibition Hall A-B #E-2508 to learn more about when and why we see such catastrophic overtraining.
Training with more data = better LLMs, right? 🚨 False! Scaling language models by adding more pre-training data can decrease your performance after post-training! Introducing "catastrophic overtraining." 🥁🧵+arXiv 👇 1/9
Join us at the AI Safety Social at #ICML2025! We'll open with a panel on the impacts of reasoning & agency on safety with @AdtRaghunathan, @ancadianadragan, @DavidDuvenaud, & @sivareddyg. Come connect over snacks & drinks. 🗓️ Thursday, July 17, from 7-9 PM in West Ballroom A
Huge congratulations to Vaishnavh, Chen and Charles on the outstanding paper award 🎉 We will be presenting our #ICML2025 work on creativity in the Oral 3A Reasoning session (West Exhibition Hall C) 10 - 11 am PT. Or please stop by our poster right after @ East Exhibition…
📢 New paper on creativity & multi-token prediction! We design minimal open-ended tasks to argue: → LLMs are limited in creativity since they learn to predict the next token → creativity can be improved via multi-token learning & injecting noise ("seed-conditioning" 🌱) 1/ 🧵
Today @ChenHenryWu and I will be presenting our #ICML work on creativity in the Oral 3A Reasoning session (West Exhibition Hall C) 10 - 11 am PT Or please stop by our poster right after @ East Exhibition Hall A-B #E-2505 11am-1:30pm. (Hope you enjoy some silly human drawings!)
Does test-time scaling help in open-ended problem domains? And by how much & why? This is quite a bit of a nuanced question to answer fully. To begin studying this, w/ @danielkty96 & @AdtRaghunathan we trained LLMs to scale test-time compute for safety. …g-adaptive-reasoners-safety.github.io
the debate around "can LLMs generate novel ideas" is why we also need *minimal* settings where we can - evaluate novelty/diversity w little room for doubt, - have cheaper iterations to test algorithmic/architectural ideas & - most importantly, think clearly about LLM behavior!
Are AI scientists already better than human researchers? We recruited 43 PhD students to spend 3 months executing research ideas proposed by an LLM agent vs human experts. Main finding: LLM ideas result in worse projects than human ideas.
Thrilled to share that I'll be starting as an Assistant Professor at Georgia Tech (@ICatGT / @GTrobotics / @mlatgt) in Fall 2026. My lab will tackle problems in robot learning, multimodal ML, and interaction. I'm recruiting PhD students this next cycle – please apply/reach out!
I believe the next big test for LLMs is whether they can generate truly novel ideas in open-ended situations. We translate notions of "creativity" from cogsci into simple tasks that reveal how far today’s models fall, and how multi-token training + randomness might help.
📢 New paper on creativity & multi-token prediction! We design minimal open-ended tasks to argue: → LLMs are limited in creativity since they learn to predict the next token → creativity can be improved via multi-token learning & injecting noise ("seed-conditioning" 🌱) 1/ 🧵
Introducing FLAME-MoE: a fully open platform for Mixture-of-Experts (MoE) research. All code, data, checkpoints, training logs, and evaluation results are public—across 7 different scales. Paper: arxiv.org/abs/2505.20225 Code: github.com/cmu-flame/FLAM…
What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision:
Excited to announce MOSS, our ICML workshop focused on discoveries at small scale! We believe there's tremendous potential & creativity in research done with limited resources and would love to hear your ideas. The submission (due May 22nd) can literally be a Jupyter notebook! :)
Announcing the 1st Workshop on Methods and Opportunities at Small Scale (MOSS) at @icmlconf 2025! 🔗Website: sites.google.com/view/moss2025 📝 We welcome submissions! 📅 Paper & jupyter notebook deadline: May 22, 2025 Topics: – Inductive biases & generalization – Training…
Super excited to announce our ICML workshop on highlighting the power (and limitations?) of small-scale in the era of large-scale ML. You can submit just a Jupyter notebook, Jupyter notebook + paper, or a survey/position paper. Do submit your work and help us spread the word!
Announcing the 1st Workshop on Methods and Opportunities at Small Scale (MOSS) at @icmlconf 2025! 🔗Website: sites.google.com/view/moss2025 📝 We welcome submissions! 📅 Paper & jupyter notebook deadline: May 22, 2025 Topics: – Inductive biases & generalization – Training…
📢 Announcing our data-centric workshop at ICML 2025 on unifying data curation frameworks across domains! 📅 Deadline: May 24, AoE 🔗 Website: dataworldicml2025.github.io We have an amazing lineup of speakers + panelists from various institutions and application areas.