Thomas Larsen
@thlarsen
AI 2027
I haven't written about this much or thought it through in detail, but here are a few aspects that go into my backdrop model: (1) especially in the long-term technological limit, I expect human labor to be wildly uncompetitive for basically any task relative to what advanced…
Spearphishing PSA—looks like there's a concerted attack on AI safety/governance folks going around. Be wary of calendar links via DM, and *never* give a 2-factor auth code over the phone. I almost got caught by this—got a phone call last week, but figured out it was sus. 🧵
And @ajeya_cotra's account has been hacked by the same folks - if you get messages from her asking to schedule a meeting, be very wary! She says she will never reach out to a potential grantee by Twitter, always email.
I like thinking for myself, so I try to never defer to anyone. But if I did, I'd defer to Ryan. Worth listening to, many important considerations discussed here.
Ryan Greenblatt is lead author of "Alignment faking in LLMs" and one of AI's most productive researchers. He puts a 25% probability on automating AI research by 2029. We discuss: • Concrete evidence for and against AGI coming soon • The 4 easiest ways for AI to take over •…
The main sycophancy threat model is that humans are imperfect raters, and so training AIs with human feedback will naturally lead to the AIs learning to produce outputs that look good to the human raters, but are not actually good. This is pretty clear in the AI safety…
A few people have asked me if a technical fix for AI model sycophancy is on the cards. In fact, a technical fix for sycophancy is trivial. In many cases all it would take is a tweak to the system prompt. The reason companies are struggling to get this right is not technical.…
I agree with Eli that these are important areas. But IMO the most important jobs in the world probably aren't on this list, instead, they are things like: - Starting a new org to fill a huge whole in the AI safety ecosystem. - Getting a job that could impact the overall USG…
There are many ways to use one's career to help AGI go better, here we list some of the top ones.
Since AI 2027 people have often asked us what they can do to make AGI go well. I've just published a blog post covering: (a) What a prepared world would look like (b) Learning recommendations to get up to speed (c) High-impact jobs and non-professional activities
Want to get up to speed on AI? My top recommendations are: - AI 2027 - Situational Awareness - AGI Ruin: A List of Lethalities - OpenAI Email Archives (from Musk v Altman) - Binging all the AI-related Dwarkesh podcast episodes.
Best Congressional AI hearing so far IMO. Great questions all around. I appreciated this one in particular, which was focused on the core issue of automated AI R&D.
We must urgently assess how far Chinese AI systems have come—and work with U.S. industry to contain the risks of automated AI R&D. Because once an AI starts improving itself, the race changes entirely.
Great post on one of the AI 2027 TTXs! I strongly agree with "The biggest threat to a rogue AI is … other AI?".
New post! A crisis simulation changed how I think about AI risk
I had a lot of fun chatting with Rob about METR's work. I stand by my claims here that the world is not on track to keep risk from AI to an acceptable level, and we desperately need more people working on these problems.
AI models currently have a 50% chance of doing something that takes a human expert one hour. This doubles every 7 months. In 2 years? They could automate full workdays. In 4 years? A full month. I discuss the most important graph in AI today with Beth Barnes, the CEO of METR,…
This claim from "AI as a normal technology" is clearly wrong, and I'm disappointed that it has gotten so much traction. 1. A lower bound for the capabilities of ASI is like a human, but sped up by a factor of 100x and working 24/7. 2. This would already clearly be…
Lots of people in AI, and especially AI policy, seem to think that aligning superintelligence is the most important issue of our time, and that failure could easily lead to extinction -- like what happened in AI 2027. But they don’t mention this fact in public because it sounds…
Obviously correct take. "but bottlenecks" is just the latest in a long line of cope.
ASI adoption will be very quick because ASI will be able to find ways to speed up the process, and the incentives will be much stronger than hiring a slightly better employee.
A clarification about what I meant by "scenario": - events attached to a timeline - descriptions of what the most important things going on in AI at each point - all the way from now until superintelligence I genuinely would love to see the Gary Marcus scenario + think it would…
This post below, complaining that skeptics haven’t made detailed predictions re a particular year, does not seem fair. I have made more detailed, public predictions about AI than anyone else. Most of them have been correct. (I will soon summarize them at @metaculus, with…