Simon LC
@simonlc_
research scientist @ the AI institute robotics & optimization phd @ stanford
I am really happy to finally share this work! Huge congrats to @albert_h_li and @pdculbert for the effort they put in making the tool hackable and interactive. Try it!🐍 pip install judo-rai
🥋 We're excited to share judo: a hackable toolbox for sampling-based MPC (SMPC), data collection, and more, designed to make it easier to experiment with high-performance control. Try it: pip install judo-rai
ICYMI: For #CoRL2024 we released a dataset of 3.5M (!) dexterous grasps, with multi-trial labels and perceptual data for 4.3k objects. Our takeaways: scale matters, and refining grasps > better sampling. Hoping our data can enable more vision-based grasps in hardware!
There have been many recent big grasping datasets, but few demos of real-world grasping using generative models. How do we achieve this? Introducing: Get a Grip (#corl2024)! We show that instead of generative models, discriminative models can attain sim2real transfer! 👀🧵👇
Excited to share our new📰, DROP: Dexterous Reorientation via Online Planning! Overview: 🔹We tackle cube rotation🧊♻️on hardware 🔹DROP is the first 🧊♻️sampling-based MPC demo. No reinforcement learning! 🔹Median 30.5 rotations w/o dropping, max of 81👑🦾 See 🧵below👇
📢Excited to share our #ICRA2023 work on robotic table wiping via RL + optimal control! 📖 arxiv.org/abs/2210.10865 🎥 youtu.be/inORKP4F3EI 💡RL (for high-level planning) + trajectory optimization (for precise control) can solve complex tasks without on-robot data collection ⬇️
Our new paper shows how task representations learned via temporal alignment enable compositional generalization for conditional policies. This allows robots to solve compound tasks by implicitly decomposing them into subtasks.
Current robot learning methods are good at imitating tasks seen during training, but struggle to compose behaviors in new ways. When training imitation policies, we found something surprising—using temporally-aligned task representations enabled compositional generalization. 1/
Learning bimanual, contact-rich robot manipulation policies that generalize over diverse objects has long been a challenge. Excited to share our work: Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation! glide-manip.github.io 🧵1/n
We're presenting Jacta: a versatile planner for learning dexterous and whole-body manipulation this week at CoRL! website jacta-manipulation.github.io paper arxiv.org/abs/2408.01258
Our team is presenting work at the Conference on Robot Learning, @corl_conf, in Munich, Germany this week! Learn more about our accepted research — theaiinstitute.com/news/corl-roun…
Really cool work on bimanual dexterity!!
Achieving bimanual dexterity with RL + Sim2Real! toruowo.github.io/bimanual-twist/ TLDR - We train two robot hands to twist bottle lids using deep RL followed by sim-to-real. A single policy trained with simple simulated bottles can generalize to drastically different real-world objects.
Excited to share our latest work, Vision-Language Frontier Maps – a SOTA approach for semantic navigation in robotics. VLFM enables robots to navigate and find objects in novel environments using vision-language foundation models, zero-shot! Accepted to #ICRA2024! 🧵
want your next robot 🤖 to move like your favorite pet 🐶? youtube videos might be all you need. checkout our recent work -- SLoMo paper: rexlab.ri.cmu.edu/papers/slomo.p… website: slomo-www.github.io/website/ video: youtu.be/bvoM-nBd7lM?si…
Some screen-captures of MuJoCo MPC's Quadruped task. 100% open-source, go play! youtu.be/esLuwaWz4oE
Introducing 𝗥𝗼𝗯𝗼𝗣𝗶𝗮𝗻𝗶𝘀𝘁 🎹🤖, a new benchmark for high-dimensional robot control! Solving it requires mastering the piano with two anthropomorphic hands. This has been one year in the making, and I couldn’t be happier to release it today! Some highlights below:
What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website: palm-e.github.io
@kevintracy48 presenting our entry guidance work @aiaa #AIAASciTech