Bo Liu (Benjamin Liu)

We've always been excited about self-play unlocking continuously improving agents. Our insight: RL selects generalizable CoT patterns from pretrained LLMs. Games provide perfect testing grounds with cheap, verifiable rewards. Self-play automatically discovers and reinforces…

We are launching a new benchmark for human-AI coordination 🙎🤖 in the Hanabi card game. 🎇🎆 If you are interested in developing methods that allow agents to collaborate w/ and support humans in complex partially observable tasks this is just for you. Crucially, we designed…

Learning GSPO proposed by Qwen team: fig 1. they propose to use sequence likelihood for importance sampling fig 2. but from the RL course by @svlevine, this is the original form of off-policy PG fig 3. per-token IS in (Dr) GRPO is an approximation of it Am I missing anything?

Pleased to share that our Multi-Turn Interactions in LLMs workshop at NeurIPS 2025! …shop-multi-turn-interaction.github.io Welcome work on Multi-Turn RL, multi-turn human<->agent/agent<->agent/agent<->environment interactions, multi-turn tool use, multi-turn alignment, multi-turn evaluation,…

🤔Long-horizon tasks: How to train LLMs for the marathon?🌀 Submit anything on 🔁"Multi-turn Interactions in LLMs"🔁 to our @NeurIPSConf workshop by 08/22: 📕 Multi-Turn RL ⚖️ Multi-Turn Alignment 💬 Multi-Turn Human-AI Teaming 📊 Multi-Turn Eval ♾️You name it! #neurips #LLM

Very excited to share that an advanced version of Gemini Deep Think is the first to have achieved gold-medal level in the International Mathematical Olympiad! 🏆, solving five out of six problems perfectly, as verified by the IMO organizers! It’s been a wild run to lead this…

Join us at #NeurIPS2025 workshop to explore the future of multi-turn AI interactions! We welcome submissions on RL for agents, alignment, evaluation methods, and more.

🚀 Call for Papers — @NeurIPSConf 2025 Workshop Multi-Turn Interactions in LLMs 📅 December 6/7 · 📍 San Diego Convention Center Join us to shape the future of interactive AI. Topics include but are not limited to: 🧠 Multi-Turn RL for Agentic Tasks (e.g., web & GUI agents,…

Today, we at @OpenAI achieved a milestone that many considered years away: gold medal-level performance on the 2025 IMO with a general reasoning LLM—under the same time limits as humans, without tools. As remarkable as that sounds, it’s even more significant than the headline 🧵

ChatGPT can now do work for you using its own computer. Introducing ChatGPT agent—a unified agentic system combining Operator’s action-taking remote browser, deep research’s web synthesis, and ChatGPT’s conversational strengths.

We have exactly the same claim in our natural language RL arxiv.org/abs/2411.14251. I cannot believe even the phrases are so similar. I believe this is what differs traditional RL and learning from experience -- we should learn more from experience, not just reward.

Good blog on "era of exploration" - Data scarcity is the new bottleneck. LLMs consume data far faster than humans can produce it. We're running out of high-quality training data. - Pretraining solved exploration by accident. Pretraining effectively pays a massive, upfront…

SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning Author's Explanation: x.com/Benjamin_eecs/… Overview: SPIRAL introduces a self-play framework for LLMs to develop reasoning skills through multi-turn, zero-sum games,…

More and better data for general reasoning beyond math? NaturalThoughts outperforms OpenThoughts3, LIMO, S1k, etc. on GPQA-Diamond, SuperGPQA, and MMLU Pro. Download 1M+ reasoning prompts: huggingface.co/datasets/faceb…

LLM + RL + Self-Play + Game = ♾ Infinity Possibility ♾ 💥New Paper Alert💥 🔗Paper: huggingface.co/papers/2506.24… 🔗Code: github.com/spiral-rl/spir…

Facebook AI Research (FAIR) is a small, prestigious lab in Meta. We don't train large models like GenAI or MSL, so it's natural that we have limited GPUs. GenAI or MSL's success or failure, past or future, doesn't reflect the work of FAIR. It is important to make this distinction

🌿Introducing NaturalThoughts 🌿 arxiv.org/abs/2507.01921 🎯 Data curation for general reasoning capabilities is still relatively underexplored. - We systematically compare different metrics for selecting high-quality and diverse reasoning traces in terms of data efficiency in…

SPIRAL enables LLMs to learn sophisticated reasoning through self-play on zero-sum games, entirely without human supervision. This groundbreaking framework creates an infinite, self-improving curriculum for autonomous AI development.

SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning

Training LLMs with self-play RL on Kuhn Poker improves math reasoning by 8.7% average.👇

We are so back