Ranjay Krishna
@RanjayKrishna
Assistant Professor, University of Washington Research Lead for Computer Vision @ Allen Institute for Artificial Intelligence
It's time for us to fundamentally re-think video encoding. It's wild that we still scale video tokens linearly with time—even if nothing is happening in the video! We're flipping that idea: what if video tokens scaled with "complexity" instead? More action, more tokens. Less…
Having trouble dealing with the excessive token number when processing a video? Check out our paper that is accepted by ICCV 2025 with an average score of 5.5! We tokenize video with tokens grounded in trajectories of all objects rather than fix-sized patches. Trained with a…
Tokenization kickstarts every Transformer pipeline—shaping how models digest data. Our latest work introduces semantic, grounded video tokenization, leveraging objectness cues to boost efficiency and performance of video understanding models.
Having trouble dealing with the excessive token number when processing a video? Check out our paper that is accepted by ICCV 2025 with an average score of 5.5! We tokenize video with tokens grounded in trajectories of all objects rather than fix-sized patches. Trained with a…
🥳 Gap year update: I'll be joining @allen_ai/@UW for 1 year (Sep2025-Jul2026 -> @JHUCompSci) & looking forward to working with amazing folks there, incl. @RanjayKrishna, @HannaHajishirzi, Ali Farhadi. 🚨 I’ll also be recruiting PhD students for my group at @JHUCompSci for Fall…
Sharing some personal updates 🥳: - I've completed my PhD at @unccs! 🎓 - Starting Fall 2026, I'll be joining the Computer Science dept. at Johns Hopkins University (@JHUCompSci) as an Assistant Professor 💙 - Currently exploring options + finalizing the plan for my gap year (Aug…
“Technical computer science savvy and deep philosophical commitments”: @UW #UWAllen alum @andreiskiii was named the @UWArtSci Dean’s Medalist in Social Sciences for his campus leadership and research contributions spanning #AI and philosophy. #UWdiscovers artsci.washington.edu/news/2025-06/2…
Large foundation models trained on massive datasets have revolutionized #AI. Supported by a @GoogleResearch Ph.D. Fellowship, @UW #UWAllen’s @cydhsieh aims to make the process more efficient and affordable to democratize AI development. #UWdiscovers news.cs.washington.edu/2025/07/09/all…
What if the secret to improving an LLM isn’t with better data or a better teacher? The Delta Learning Hypothesis: learn from the **delta** in performance between two weaker LLMs to instruction-tune a stronger one.
🤔 How do we train AI models that surpass their teachers? 🚨 In #COLM2025: ✨Delta learning ✨makes LLM post-training cheap and easy – with only weak data, we beat open 8B SOTA 🤯 The secret? Learn from the *differences* in weak data pairs! 📜 arxiv.org/abs/2507.06187 🧵 below
🤔 How do we train AI models that surpass their teachers? 🚨 In #COLM2025: ✨Delta learning ✨makes LLM post-training cheap and easy – with only weak data, we beat open 8B SOTA 🤯 The secret? Learn from the *differences* in weak data pairs! 📜 arxiv.org/abs/2507.06187 🧵 below
Having trouble dealing with the excessive token number when processing a video? Check out our paper that is accepted by ICCV 2025 with an average score of 5.5! We tokenize video with tokens grounded in trajectories of all objects rather than fix-sized patches. Trained with a…
“Spatial thinking is the foundation of thought, moving in spaces essential to life." - Barbara Tversky in Mind in Motion. Spatial reasoning goes beyond just reasoning about what is around you. It's reasoning about dynamics, actions, egocentric motions, and much more. Test…
Thrilled to announce that our paper SAT has been accepted to #COLM2025! 🎉 Better yet— all the data and code are already open-sourced. Dive in, experiment, and let us know what you build! Data:huggingface.co/datasets/array… Code:github.com/arijitray1993/…
@COLM_conf decisions are out, and so are we The strength of submissions this year amazed us! Many many hard decisions 😩 + @AdtRaghunathan, @eunsolc, @RanjayKrishna 😴😴😴
Congrats to my incredible PhD advisor!! Every time I write a paper or give a talk, I hear Michael's voice in my head, questioning every sentence (including this one). The academic ghost I never knew I needed 👻📚
Thank you to everyone for your energy and enthusiasm in joining this adventure with me so far!
It’s incredible to have both your advisors at the same company! With @fox_dieter17849 building the Robotics team, and @RanjayKrishna leading PRIOR, @allen_ai is set to become a powerhouse in robotics, computer vision, and embodied AI for open science research . Excited to be part…
Talent density only going up and to the right at ai2. Let's keep pushing.
Can VLMs build Spatial Mental Models like humans? Reasoning from limited views? Reasoning from partial observations? Reasoning about unseen objects behind furniture / beyond current view? Check out MindCube! 🌐mll-lab-nu.github.io/mind-cube/ 📰arxiv.org/pdf/2506.21458…
[FAccT 2025 paper!] 🧵 Remember the March 2024 Gemini controversy? People were upset when it generated "diverse" but historically inappropriate images—like Black Founding Fathers and female Asian Nazis. This sparked our research... 1/6
Very excited to see this new startup finally online. Very cool demos.
Today we're excited to share a glimpse of what we're building at Generalist. As a first step towards our mission of making general-purpose robots a reality, we're pushing the frontiers of what end-to-end AI models can achieve in the real world. Here's a preview of our early…
I’m thrilled to share that I will be joining Johns Hopkins University’s Department of Computer Science (@JHUCompSci, @HopkinsDSAI) as an Assistant Professor this fall.
Yayyy!!! Best paper honorable mention at CVPR goes to our Molmo and Pixmo @allen_ai! This is now becoming a tend :) Last year both OLMo and Dolma received best paper awards at ACL.
I’m excited to present Synthetic Visual Genome with James tomorrow afternoon! 🥳 Stop by poster #354 if you’re interested in / wanna chat about synthetic data, scene graphs, multimodal LMs, and more😎
🔥We are excited to present our work Synthetic Visual Genome (SVG) at #CVPR25 tomorrow! 🕸️ Dense scene graph with diverse relationship types. 🎯 Generate scene graphs with SAM segmentation masks! 🔗Project link: bit.ly/4e1uMDm 📍 Poster: #32689, Fri 2-4 PM 👇🧵
🔥We are excited to present our work Synthetic Visual Genome (SVG) at #CVPR25 tomorrow! 🕸️ Dense scene graph with diverse relationship types. 🎯 Generate scene graphs with SAM segmentation masks! 🔗Project link: bit.ly/4e1uMDm 📍 Poster: #32689, Fri 2-4 PM 👇🧵
Excited to share our work Aurora at #CVPR2025! 🗓️ Tomorrow ( Friday June 13th) 🖼️ Poster #349 📍 Exhibit Hall D 🕥 10:30 AM – 12:30 PM Come by to chat and see what we’ve been working on! ✨
Introducing AURORA 🌟: Our new training framework to enhance multimodal language models with Perception Tokens; a game-changer for tasks requiring deep visual reasoning like relative depth estimation and object counting. Let’s take a closer look at how it works.🧵[1/8]