Adithya Bhaskar

Ever wished circuit finding was more precise, efficient, and scalable? Now it is! In our new preprint, we propose Edge Pruning, a conceptually simple yet effective way to find circuits in models. Details in 🧵! Work done with @_awettig @danfriedman0 @danqi_chen 1/6

Are AI scientists already better than human researchers? We recruited 43 PhD students to spend 3 months executing research ideas proposed by an LLM agent vs human experts. Main finding: LLM ideas result in worse projects than human ideas.

🤔 Recent mech interp work showed that retrieval heads can explain some long-context behavior. But can we use this insight for retrieval? 📣 Introducing QRHeads (query-focused retrieval heads) that enhance retrieval Main contributions: 🔍 Better head detection: we find a…

Introducing COMPACT: COMPositional Atomic-to-complex Visual Capability Tuning, a data-efficient approach to improve multimodal models on complex visual tasks without scaling data volume. 📦 arxiv.org/abs/2504.21850 1/10

Want to train large vision-language models but drowning in data? arxiv.org/abs/2501.00654 Introducing ICONS - we demonstrate how to select only 20% of training samples while maintaining 98.6% of the performance, and 60% of training samples to achieve 102.1% of the performance.

Have you ever wondered why we don’t use multiple visual encoders for VideoLLMs? We thought the same! Excited to announce our latest work MERV, on using Multiple Encoders for Representing Videos in VideoLLMs, outperforming prior works with the same data. 🧵

Past work observed that DPO often decreases the probability of preferred responses. So where does the probability go? 🧐 We investigate the causes for this counter-intuitive phenomenon and show that it can lead to surprising failures in alignment! 📰 arxiv.org/abs/2410.08847 🧵

I’m at COLM from Monday to Wednesday. Reach out if you want to chat!

Come check out our poster at #ACL2024! I will be at the 4pm poster session, stop by to chat about long-context models

I'll be at ACL 2024! I'd love to chat with about interpretability, preference optimization, science of LM, or any NLP topics -- feel free to reach out! Oh, and I'll present The Heuristic Core (arxiv.org/abs/2403.03942) both as an oral (Aug 13 10:30) and a poster (Aug 12 14:00).

How can we understand neural chatbots in terms of interpretable, symbolic mechanisms? To explore this question, we constructed a Transformer that implements the classic ELIZA chatbot algorithm (with @Abhishek_034 and @danqi_chen). Paper: arxiv.org/abs/2407.10949 (1/6)

My new blog post argues from first principles how length normalization in preference learning objectives (e.g., SimPO) can facilitate learning from model-annotated preference data. Check it out! cs.princeton.edu/~smalladi/blog…