Cheng Lu
@clu_cheng
Member of technical staff @OpenAI. PhD @Tsinghua_Uni. Interested in scalable generative models.
Excited to share our latest research progress (joint work with @DrYangSong ): Consistency models can now scale stably to ImageNet 512x512 with up to 1.5B parameters using a simplified algorithm, and our 2-step samples closely approach the quality of diffusion models. See more…
Introducing sCMs: our latest consistency models with a simplified formulation, improved training stability, and scalability. sCMs generate samples comparable to leading diffusion models but require only two sampling steps. openai.com/index/simplify…
Exact same feeling
It’s truly a privilege to be able to wake up every morning, see where the latest intelligence frontier is, and help push it a little further.
Congrats! This is an incredible milestone and I was truly shocked by it. “Thinking for hours” means 10x or even 100x of current test-time compute, and I can’t wait to see the model think for days, months, years, centuries to solve the science challenges!
1/N I’m excited to share that our latest @OpenAI experimental reasoning LLM has achieved a longstanding grand challenge in AI: gold medal-level performance on the world’s most prestigious math competition—the International Math Olympiad (IMO).
I didn't want to post on Grok safety since I work at a competitor, but it's not about competition. I appreciate the scientists and engineers at @xai but the way safety was handled is completely irresponsible. Thread below.
So true
Why you should stop working on RL research and instead work on product // The technology that unlocked the big scaling shift in AI is the internet, not transformers I think it's well known that data is the most important thing in AI, and also that researchers choose not to work…
A very promising direction for real-time video generation! arxiv.org/abs/2506.01380 nextframed.github.io 1. You can always use DPM-Solver++ to accelerate your flow matching model. 2. sCM can even scale to video diffusion model and boost the sample quality a lot!
So true
I’ve changed so little. From my 1978 Bachelor’s thesis: “The adult human mind is very complex, but the question remains open whether the learning processes that constructed it in interaction with the environment are similarly complex. Much evidence and many peoples’ intuitions…
I’ve changed so little. From my 1978 Bachelor’s thesis: “The adult human mind is very complex, but the question remains open whether the learning processes that constructed it in interaction with the environment are similarly complex. Much evidence and many peoples’ intuitions…
if you want to see someone truly passionate about image generation, look no further than @gabeeegoooh he lives and breathes making image generation wonderful.
this is ready now for the world
I closely worked with and learned a lot from @gabeeegoooh and @ajabri , and it can be summarized in the old saying: be the change you want to see
this is ready now for the world
Congrats on everyone who worked in the GPT-4o image generation team! It’s really impressive and I’m so proud of us! It was also quite enjoyable working with such a group of talented people!
Lots of fun w/ @gabeeegoooh @eliza_luth @ajabri @kenjihata @jianfw @yapdianang @prafdhar
Still think consistency models are bad at scale? In fact, sCM can be stably scaled to modern text-to-image diffusion models and greatly improve the generation speed and 1-step generation quality!
🚀🔥SANA-Sprint: One-step text-to-image generation in 0.1s with SANA! arxiv.org/abs/2503.09641 SANA-Sprint delivers: ⚡️ 1024×1024 images in 0.1s on H100 🏆 SOTA: 7.59 FID in ONE step ⚙️ 10x faster than FLUX-Schnell 💻 Deployable on laptop (0.31s on RTX 4090) Code coming soon!🥳
Since the behaviors of consistency models are quite different in pixel and latent spaces, I wonder if using these new AEs can further improve the training of consistency models
If you want to diffuse stuff, its frequency behaviour is important🌊 (sander.ai/2024/09/02/spe…). For latents, you can shape the spectrum! Like EQ-VAE, they find: equivariance ⇒ better latents. Loving all the recent work on tweaking latents, might be time for another blog post✍️
🚀【Large Language Diffusion Models】#DiffusionModels #LLM #LLaDA We built LLaDA-8B—the FIRST non-autoregressive model rivaling LLaMA3! CRUSHES Llama2-7B on ~20 tasks while unlocking ICL/instruction-following/multi-turn chat