Han Wang

🚨Real-world retrieval is messy: queries can be ambiguous, or documents may conflict/have incorrect/irrelevant info. How can we jointly address all these problems? We introduce: ➡️ RAMDocs, a challenging dataset with ambiguity, misinformation, and noise. ➡️ MADAM-RAG, a…

🎉 Our paper, GenerationPrograms, which proposes a modular framework for attributable text generation, has been accepted to @COLM_conf! GenerationPrograms produces a program that executes to text, providing an auditable trace of how the text was generated and major gains on…

🎉 Glad to see our work on handling conflicting & noisy evidence and ambiguous queries in RAG systems (via a new benchmark & multi-agent debate method) has been accepted to #COLM2025 @COLM_conf!! 🇨🇦 Congrats to Han on leading this effort. More details in the thread below and…

🥳 Excited to share our work -- Retrieval-Augmented Generation with Conflicting Evidence -- on addressing conflict in RAG due to ambiguity, misinformation, and noisy/irrelevant evidence has been accepted to @COLM_conf #COLM2025! Our new benchmark RAMDocs proves challenging for…

🚨Introducing Video-RTS: Resource-Efficient RL for Video Reasoning with Adaptive Video TTS! While RL-based video reasoning with LLMs has advanced, the reliance on large-scale SFT with extensive video data and long CoT annotations remains a major bottleneck. Video-RTS tackles…

🥳Excited to share that I’ll be joining @unccs as postdoc this fall. Looking forward to work with @mohitban47 & amazing students at @unc_ai_group. I'll continue working on retrieval, aligning knowledge modules with LLM's parametric knowledge, and expanding to various modalities.

🥳 Excited to share that I’ll be joining the CS Department at UNC-Chapel Hill (@unccs @unc_ai_group) as an Assistant Professor starting Fall 2026! Before that, I’ll be working at Ai2 Prior (@allen_ai @Ai2Prior) and UW (@uwcse) on multimodal understanding and generation.

🎉 Excited to share that CAPTURe has been accepted to #ICCV2025! CAPTURe is a new benchmark for VLM reasoning that requires completing patterns to count objects that are occluded from view. We find that SOTA VLMs struggle with both counting and reasoning about partial patterns!…

🎉Excited to announce VEEGIE has been accepted to #ICCV2025 ! VEGGIE is a unified MLLM + Diffusion framework for instructional video editing. It presents a systematic approach spanning data, model, benchmark, and evaluation design, and shows strong multi-skill editing +…

New paper Alert 🚨 Introducing MEXA: A general and training-free multimodal reasoning framework via dynamic multi-expert skill selection, aggregation and deep reasoning! MEXA: 1. Selects task- and modality-relevant experts based on the query and various required multimodal…

🚨 Want models to better utilize and ground on the provided knowledge? We introduce Context-INformed Grounding Supervision (CINGS)! Training LLM with CINGS significantly boosts grounding abilities in both text and vision-language models compared to standard instruction tuning.

Excited to share GenerationPrograms! 🚀 How do we get LLMs to cite their sources? GenerationPrograms is attributable by design, producing a program that executes text w/ a trace of how the text was generated! Gains of up to +39 Attribution F1 and eliminates uncited sentences,…

Thanks for the discovering + sharing our work on contextualized late-interaction based multimodal content retrieval, Omar! (and ColBERT is awesome of course) 😀

Wow I missed this extra fancy ColBERT model. > A late-interaction retriever which jointly encodes/contextualizes information from many modalities, allowing for fine-grained matching between the query and implicitly finding the most relevant modality.

Excited to present VideoTree🌲 at #CVPR2025 Fri at 10:30AM! VideoTree improves long-video QA via smart sampling: -Query-adaptive: finds the parts of the video relevant to the query -Coarse-to-fine structure: structured hierarchically to sample granularly from relevant segments

Introducing CLaMR -- a late-interaction retriever for complex multimodal video content! 📽️📚 ➡️ Jointly encodes frames, speech, on-screen text, and metadata to answer diverse queries grounded across modalities ➡️ Trained with a new dataset we introduce, MultiVENT 2.0++, a…

How can a multimodal retriever accurately retrieve docs from massive online video content that spans multiple modalities? We introduce CLaMR, a contextualized late-interaction retriever that jointly encodes all modalities and dynamically selects those containing the relevant…

Excited to announce CLaMR, our new retriever for multimodal documents! Strong performance improvements (+25 nDGC@10) compared to both multimodal and unimodal retrieval baselines. 🤝 CLaMR jointly encodes multiple modalities and selects the most relevant ones for each query. 🏋️‍♂️…

Excited to share our new work, CLaMR! 🚀 We tackle multimodal content retrieval by jointly considering video, speech, OCR, and metadata. CLaMR learns to dynamically pick the right modality for your query, boosting retrieval by 25 nDCG@10 over single modality retrieval! 🧐…

Excited to share Video-Skill-CoT🎬🛠️– a new framework for domain-adaptive video reasoning with skill-aware Chain-of-Thought (CoT) supervision! ⚡️Key Highlights: ➡️ Automatically extracts domain-specific reasoning skills from questions and organizes them into a unified taxonomy,…