Ilya Sutskever
@ilyasut
SSI @SSI
I sent the following message to our team and investors: — As you know, Daniel Gross’s time with us has been winding down, and as of June 29 he is officially no longer a part of SSI. We are grateful for his early contributions to the company and wish him well in his next…
And congratulations to @demishassabis and John Jumper for winning the Nobel Prize in Chemistry!!
Congratulations to @geoffreyhinton for winning the Nobel Prize in physics!!
Mountain: identified. Time to climb
SSI is building a straight shot to safe superintelligence. We’ve raised $1B from NFDG, a16z, Sequoia, DST Global, and SV Angel. We’re hiring: ssi.inc
We're announcing, together with @ericschmidt: Superalignment Fast Grants. $10M in grants for technical research on aligning superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more. Apply by Feb 18! openai.com/blog/superalig…
RLHF works great for today's models. But aligning future superhuman models will present fundamentally new challenges. We need new approaches + scientific understanding. New researchers can make enormous contributions—and we want to fund you! Apply by Feb 18!
We're announcing, together with @ericschmidt: Superalignment Fast Grants. $10M in grants for technical research on aligning superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more. Apply by Feb 18! openai.com/blog/superalig…
My view is that what makes super-alignment "super" is ensuring we can safely scale the capabilities of AIs even though we can't scale their human supervisors. For this, it is imperative to study the "weak teacher strong student" setting. Paper shows great promise in this area!
Open AI new paper Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision paper: cdn.openai.com/papers/weak-to… blog: openai.com/research/weak-… Widely used alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on the ability of…
i'd particularly like to recognize @CollinBurns4 for today's generalization result, who came to openai excited to pursue this vision and helped get the rest of the team excited about it!
Large pretrained models have excellent raw capabilities—but can we elicit these fully with only weak supervision? GPT-4 supervised by ~GPT-2 recovers performance close to GPT-3.5 supervised by humans—generalizing to solve even hard problems where the weak supervisor failed!
new paper! one reason aligning superintelligence is hard is because it will be different from current models, so doing useful empirical research today is hard. we fix one major disanalogy of previous empirical setups. I'm excited for future work making it even more analogous.
In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-…
New direction for AI alignment — weak-to-strong generalization. Promising initial results: we used outputs from a weak model (fine-tuned GPT-2) to communicate a task to a stronger model (GPT-4), resulting in intermediate (GPT-3-level) performance.
In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-…
Extremely excited to have this work out, the first paper from the Superalignment team! We study how large models can generalize from supervision of much weaker models.
In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-…
Kudos especially to @CollinBurns4 for being the visionary behind this work, @Pavel_Izmailov for all the great scientific inquisition, @ilyasut for stoking the fires, @janhkirchner and @leopoldasch for moving things forward every day. Amazing ✨
Super excited about our new research direction for aligning smarter-than-human AI: We finetune large models to generalize from weak supervision—using small models instead of humans as weak supervisors. Check out our new paper: openai.com/research/weak-…
In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-…