Ruchit Rawal

Introducing ARGUS 👁️ A benchmark for measuring hallucinations and omissions in free-form captions generated by Video-LLMs.

Updates from our depth-recurrent model adventure: 📈 KV cache sharing across recurrences = higher accuracy quicker + less memory 🔧 Now with vLLM integration & finetuning examples! github.com/seal-rg/recurr…

(Structured) Model pruning is a nice tool when you really need to deploy a model that is a *bit* smaller, but don't want to deploy a bigger hammer like quantization. We recently published an improved *automated* model pruning method, surprisingly based on model merging:

Test cricket at its finest.

🚨Announcing Zebra-CoT, a large-scale dataset of high quality interleaved image-text reasoning traces 📜. Humans often draw visual aids like diagrams when solving problems, but existing VLMs reason mostly in pure text. 1/n

Many VLMs claim to process hours of video. But can they follow the story?🤔 Today, we introduce TimeScope: The benchmark that separates true temporal understanding from marketing hype. Let's see how much VLMs really understand!⏳

🤖 Transformers can write poetry, code, and generate stunning art, but can they predict seemingly random numbers? We show that they learn to predict simple PRNGs (LCGs) by figuring out prime factorization on their own!🤯 Find Darshil tomorrow, 11am at #ICML2025 poster session!

Introducing MORSE-500 🌐 morse-500.github.io 500 scripted videos that stress-test six reasoning skills — beyond math, beyond static pics, built to get harder. Key Features: 🚀 Fresh & Portable 🎯 Diverse Categories 👁️ Pure Visual Cues 📈 Scalable Difficulty Dive in 🧵

🎉ArgusBench is accepted to ICCV 2025!! 🌊 It's a new benchmark for evaluating hallucinations & omissions in Video-LLM dense captions! - Unlike QA-based metrics, we focus on open-ended text generation. Why? Verification ≠ Generation! - How it works: We match generated…

ARGUS 👁️ has set its sights on Hawaii! Catch us at @ICCVConference this fall 🏖️

We broke @cluely's “Cheat on Everything” tool… using an audio prompt injection What happened next was pure gold Watch the full video 👇

Most papers discuss the hallucination problem in visual language models. In this paper, we present a framework to quantify both hallucination and omission problems in modern video LLMs. Both dataset and benchmarking code out!