Sahil Verma
@Sahil1V
PhD student @uwcse. Robustness and Interpretability. Currently at @MSFTResearch. Former intern at @amazon, @itsArthurAI. Undergrad @IITKanpur
๐จ Code is live! Check out LoRe โ a modular, lightweight codebase for personalized reward modeling from user preferences. ๐ฆ Few-shot personalization ๐ Benchmarks: TLDR, PRISM, PersonalLLM ๐ github.com/facebookresearโฆ Huge thanks to @AIatMeta for open-sourcing this research ๐
๐ง Your LLM should model how you think, not reduce you to preassigned traits ๐ข Introducing LoRe: a low-rank reward modeling framework for personalized RLHF โ Demographic grouping/handcrafted traits โ Infers implicit preferences โ Few-shot adaptation ๐ arxiv.org/abs/2504.14439
Llama Nemotron model just got Super-Charged โก๏ธWe released Llama-Nemotron-Super-v1.5 today! The best open model that can be deployed on a single H100 ๐ Enhanced for reasoning, tool use, general chat, and instruction following. HF : huggingface.co/nvidia/Llama-3โฆ
Very excited to announce Llama-Nemotron-Super-V1.5! Super-V1.5 is now better than Ultra-V1. This is currently the best model that can be deployed on a single H100. Reasoning On/Off and drop in replacement for V1. Open-weight, code and data on HF huggingface.co/nvidia/Llama-3โฆ
I will be at the Actionable Interpretability Workshop (@ActInterp, #ICML) presenting *SSAEs* in the East Ballroom A from 1-2pm. Drop by (or send a DM) to chat about (actionable) interpretability, (actionable) identifiability, and everything in between!
1\ Hi, can I get an unsupervised sparse autoencoder for steering, please? I only have unlabeled data varying across multiple unknown concepts. Oh, and make sure it learns the same features each time! Yes! A freshly brewed Sparse Shift Autoencoder (SSAE) coming right up. ๐งถ
Transformers struggle with length generalization and long context. What can we do about it? Our new #TMLR paper with @rolandalong , @paul_smolensky and @JianfengGao0217 shows how to handle the issue. Using a new attention mechanism called TRA. Curious? Read the ๐งต for more ๐ค
Are you compositionally curious ๐ค Want to know how to learn embeddings using๐ฒ? In our new #ICML2025 paper, we present Banyan: A recursive net that you can train super efficiently for any language or domain, and get embeddings competitive with much much larger LLMs 1/๐งต
๐ตโ๐ซ Struggling with ๐๐ข๐ง๐-๐ญ๐ฎ๐ง๐ข๐ง๐ ๐๐จ๐? Meet ๐๐๐ง๐ฌ๐๐๐ข๐ฑ๐๐ซ โ an MoE post-training method that offers more ๐ฉ๐ซ๐๐๐ข๐ฌ๐ ๐ซ๐จ๐ฎ๐ญ๐๐ซ ๐ ๐ซ๐๐๐ข๐๐ง๐ญ, making MoE ๐๐๐ฌ๐ข๐๐ซ ๐ญ๐จ ๐ญ๐ซ๐๐ข๐ง and ๐๐๐ญ๐ญ๐๐ซ ๐ฉ๐๐ซ๐๐จ๐ซ๐ฆ๐ข๐ง๐ ! Blog: fengyao.notion.site/moe-posttrainiโฆโฆ
Using retrieval? --> check out this work by my awesome collaborator on how to increase diversity when retrieving!
1/8 ๐ How can retrieval augmentation be made both relevant and non-redundant for few-shot adaptation? I'm excited to introduce COBRA. Catch our poster at #CVPR25 (ExHall D, Poster #450) on Sat 14 Jun, 5โ7 p.m. CDT: cvpr.thecvf.com/virtual/2025/pโฆ
๐ฅ "Vibe coding" is everywhereโbut is it really care-free? We introduce ๐๐๐๐, an RL framework that trains LLMs with automated program analysis feedback, enabling "vibe coding" to be not just fastโbut ๐ฏ๐ฎ๐ฅ๐ง๐๐ซ๐๐๐ข๐ฅ๐ข๐ญ๐ฒ-๐๐ซ๐๐ & ๐ฉ๐ซ๐จ๐๐ฎ๐๐ญ๐ข๐จ๐ง-๐ซ๐๐๐๐ฒ ๐ก๏ธโฆ
Introducing soarXiv โ๏ธ, the most beautiful way to explore human knowledge Take any paper's URL and replace arxiv with soarxiv (show in video) to teleport to its place in the universe I've embedded all 2.8M papers up until April 2025 Try it at: soarxiv dot org
โกโก Llama-Nemotron-Ultra-253B just dropped: our most advanced open reasoning model ๐งต๐
โก๏ธ Llama-Nemotron-Ultra is fully open โ weights and post-training data. Achieves 76.0% on GPQA via FP8 RL training with GRPO. Best open model for scientific reasoning ๐ x.com/soumyesinghal/โฆ
We are excited to release Llama-Nemotron-Ultra! This is a reasoning ON/OFF, dense 253B model. Open weights and post-training data. huggingface.co/nvidia/Llama-3โฆ We started with llama-405B, changed it via NAS pruning then followed by reasoning-focused post-training: SFT + RL in FP8.
๐ Meet Llama-Nemotron-Super-49B, our teamโs new reasoning model released at #GTC25! Proud to have contributed ๐ง . Optimized via NAS for single-GPU inference, it delivers impressive reasoning performance at 49B scale with reasoning mode control (ON/OFF). huggingface.co/nvidia/Llama-3โฆ
We are excited to release new Llama-Nemotron models. These models allow you to set reasoning ON/OFF during runtime. We also release all the post-training data under CC-BY-4! Try it now on build.nvidia.com/nvidia/llama-3โฆ HF collection: huggingface.co/collections/nvโฆ
Come for the ridiculous 30 column spreadsheet created at @sweetgreen, stay for a critical discussion of how you *actually* scale models.
We nearly drove ourselves insane trying to reproduce scaling laws papers ๐ So of course we wrote a paper about itย ๐ตโ๐ซ 1/9
1\ Hi, can I get an unsupervised sparse autoencoder for steering, please? I only have unlabeled data varying across multiple unknown concepts. Oh, and make sure it learns the same features each time! Yes! A freshly brewed Sparse Shift Autoencoder (SSAE) coming right up. ๐งถ
๐ค LLM alignment feeling like a black box? ๐ฆ Our paper introduces Reward-Aware Preference Optimization (RPO)โlinking DPO, IPO, SimPO, REINFORCE (LOO), & more! ๐ We unpack key design choices with crisp ablations & clear insights! ๐ Led by @ssydasheng arxiv.org/abs/2502.00203
Our team put together a unified mathematical framework to analyze popular model alignment algorithms. โReward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignmentโ arxiv.org/pdf/2502.00203