Yanjie Ze
@ZeYanjie
CS PhD @Stanford. Focus on Humanoid Intelligence.
🤖Introducing TWIST: Teleoperated Whole-Body Imitation System. We develop a humanoid teleoperation system to enable coordinated, versatile, whole-body movements, using a single neural network. This is our first step toward general-purpose robots. 🌐humanoid-teleop.github.io
What’s keeping robot arms from working like human arms? They're big, slow, have the wrong joints, and can't conform to their environment. DexWrist solves all of these issues and simplifies learning constrained, dynamic manipulation👉 dexwrist.csail.mit.edu
We had a great discussion with @ZeYanjie about his new paper TWIST! Whole body teleoperation. And I do mean whole body: lifting things off the ground, holding with two hands while kicking, etc. Very cool and inspiring work that I hope leads to very useful humanoid robot data.
Full episode dropping soon! Geeking out with @ZeYanjie on TWIST: Teleoperated Whole-Body Imitation System yanjieze.com/TWIST/ Co-hosted by @chris_j_paxton & @micoolcho
Full episode dropping soon! Geeking out with @ZeYanjie on TWIST: Teleoperated Whole-Body Imitation System yanjieze.com/TWIST/ Co-hosted by @chris_j_paxton & @micoolcho
Focus on the core
I wrote a fun little article about all the ways to dodge the need for real-world robot data. I think it has a cute title. sergeylevine.substack.com/p/sporks-of-agi
Large robot datasets are crucial for training 🤖foundation models. Yet, we lack systematic understanding of what data matters. Introducing MimicLabs ✅System to generate large synthetic robot 🦾 datasets ✅Data-composition study 🗄️ on how to collect and use large datasets 🧵1/
Interesting task. When can we have a fully humanoid band? Drummer has been here.
🎶Can a robot learn to play music? YES! — by teaching itself, one beat at a time 🎼 🥁Introducing Robot Drummer: Learning Rhythmic Skills for Humanoid Drumming 🤖 🔍 For details, check out: robotdrummer.github.io
Tactile interaction in the wild can unlock fine-grained manipulation! 🌿🤖✋ We built a portable handheld tactile gripper that enables large-scale visuo-tactile data collection in real-world settings. By pretraining on this data, we bridge vision and touch—allowing robots to:…
Can an AI model predict perfectly and still have a terrible world model? What would that even mean? Our new ICML paper formalizes these questions One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵
Action chunking is a great idea in robotics: by getting a model to produce a short sequence of actions, it _just works better_ for some mysterious reason. Now it turns out this can help in RL too, and it's a bit clearer why: action chunks help explore and help with backups. 🧵👇
Let me introduce more details about our new quadruped robot, KLEIYN. This robot is capable of chimney climbing on walls of various widths, like Ninja! I will explain its design and learning methodology. Thread👇
Introducing Hierarchical Surgical Robot Transformer (SRT-H), a language-guided policy for autonomous surgery🤖🏥 On the da Vinci robot, we perform a real surgical procedure on animal tissue. Collaboration b/w @JohnsHopkins & @Stanford
TRI's latest Large Behavior Model (LBM) paper landed on arxiv last night! Check out our project website: toyotaresearchinstitute.github.io/lbm1/ One of our main goals for this paper was to put out a very careful and thorough study on the topic to help people understand the state of the…
Time for another blog post -- "The Inner Robot" -- on how our subconscious processes mislead us on the nature of human manipulation, by managing it brilliantly and quietly. Hope you find it rewarding. It is a seven-minute read. mtmason.com/the-inner-robo…
What would a World Model look like if we start from a real embodied agent acting in the real world? It has to have: 1) A real, physically grounded and complex action space—not just abstract control signals. 2) Diverse, real-life scenarios and activities. Or in short: It has to…
Details about how these motions are achieved. RL with motion reference trajectories #RSS2025
Atlas is demonstrating reinforcement learning policies developed using a motion capture suit. This demonstration was developed in partnership with Boston Dynamics and @rai_inst.
#ICCV2025 🤩3D world generation is cool, but it is cooler to play with the worlds using 3D actions 👆💨, and see what happens! — Introducing *WonderPlay*: Now you can create dynamic 3D scenes that respond to your 3D actions from a single image! Web: kyleleey.github.io/WonderPlay/ 🧵1/7
Ep#16 with @ZeYanjie on TWIST: Teleoperated Whole-Body Imitation System yanjieze.com/TWIST/ Co-hosted by @chris_j_paxton & @micoolcho
🚀 Introducing LeVERB, the first 𝗹𝗮𝘁𝗲𝗻𝘁 𝘄𝗵𝗼𝗹𝗲-𝗯𝗼𝗱𝘆 𝗵𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗩𝗟𝗔 (upper- & lower-body), trained on sim data and zero-shot deployed. Addressing interactive tasks: navigation, sitting, locomotion with verbal instruction. 🧵 ember-lab-berkeley.github.io/LeVERB-Website/
Ever imagined a robot dog that can organize shoes, collect toys, and even scoop cat litter? 😼🐾 Introducing Human2LocoMan, a unified framework for scaling up imitation learning for quadrupedal manipulation—by pretraining on human data and finetuning on robots. Webpage:…
Tesla Robotaxi: A New Era Begins I’ve (very fortunately) been part of multiple robotaxi launches. But this one is different and feels much more profound. It’s a paradigm shift. It’s the GPT moment for real-world autonomy. Tesla’s robotaxi runs vision-only -- no lidar, no radar,…
The future of transportation is here with Tesla robotaxi