Daniel Kokotajlo

"How, exactly, could AI take over by 2027?" Introducing AI 2027: a deeply-researched scenario forecast I wrote alongside @slatestarcodex, @eli_lifland, and @thlarsen

Love this @robinhanson piece. This is why @ystrickler's 'dark forest theory' of the Internet is true. We should really reward risk, originality, unorthodoxy, and weirdness more. overcomingbias.com/2023/02/why-is…

Owain et al keep doing really interesting research! I'm impressed. And I think that all these clues are eventually going to add up to a better fundamental understanding of what's going on inside these AIs.

There is still an opportunity for @OpenAI to live up to its founding promises, instead of abandoning them. Here I explain what this could look like.

chain-of-thought monitorability is a wonderful thing ;) gist.githubusercontent.com/nostalgebraist…

I haven't written about this much or thought it through in detail, but here are a few aspects that go into my backdrop model: (1) especially in the long-term technological limit, I expect human labor to be wildly uncompetitive for basically any task relative to what advanced…

having now thought about an issue for a few hours, i'm pleased to announce that once again the ingroup is entirely correct and honest, and the outgroup is wrong and unserious.

Just wanna say, C.S. Lewis' The Abolition of Man is great & I hope more people read it, especially people interested in ethics, metaethics, or futurism. I put a tiny shoutout to it in AI 2027:

Not sure what to make of this. Seems surprising that these AIs are about as good relative to humans at long tasks compared to short tasks? Seems qualitatively different from what @METR_Evals seems to be finding? Thoughts?

At Redwood Research, we recently posted a list of empirical AI security/safety project proposal docs across a variety of areas. Link in thread.

I am extremely excited about the potential of chain-of-thought faithfulness & interpretability. It has significantly influenced the design of our reasoning models, starting with o1-preview. As AI systems spend more compute working e.g. on long term research problems, it is…

When models start reasoning step-by-step, we suddenly get a huge safety gift: a window into their thought process. We could easily lose this if we're not careful. We're publishing a paper urging frontier labs: please don't train away this monitorability. Authored and endorsed…

I'm very happy to see this happen. I think that we're in a vastly better position to solve the alignment problem if we can see what our AIs are thinking, and I think that we sorta mostly can right now, but that by default in the future companies will move away from this paradigm…

A decade earlier, I tried to explain why, *even if* we didn't end up in a timeline like this, messing with machine superintelligence would *still* be dangerous. But we're not *in* the Very Careful People timeline. We're in *this* timeline. It'd be suicide plain and simple.

METR previously estimated that the time horizon of AI agents on software tasks is doubling every 7 months. We have now analyzed 9 other benchmarks for scientific reasoning, math, robotics, computer use, and self-driving; we observe generally similar rates of improvement.

Claude 4 Opus has pioneered a galaxy-brained strategy of "store unfindabillity as major strategic moat" for its merch store. From its memories:

yet another case of fearmongering about AI to hype up the industry and line the pockets of tech billionaires, this time coming from noted capitalist libertarian <checks notes> bernie sanders

if your alignment plan relies on the Internet not being stupid then your alignment plan is terrible if your alignment plan relies on the Internet not being stupid then your alignment plan is terrible if your alignment plan relies on the Internet not being stupid then your…

Grok 4 is not just plausibly SOTA but an amount of scaling never claimed before (equal parts pretraining and reasoning training) - that's wild Even more wild that the model used so much compute but capabilities don't seem out of distribution from o3 or Claude 4 Opus

Thanks Jasmine, it was great to chat. As mentioned I was eager to get off the media circuit and back to research, but you wanted to talk about something different and underexplored compared to everyone else, so I did one last podcast basically :)