Joe Carlsmith
@jkcarlsmith
Philosophy, futurism, AI. Senior advisor @open_phil. Opinions my own.
I put my report on existential risk from power-seeking AI on arXiv: arxiv.org/abs/2206.13353
In response to a comment from @herbiebradley on my recent talk, I wrote a bit about my backdrop model of the long-term role of human labor in a post-AGI economy.
I haven't written about this much or thought it through in detail, but here are a few aspects that go into my backdrop model: (1) especially in the long-term technological limit, I expect human labor to be wildly uncompetitive for basically any task relative to what advanced…
Thoughtful discussion of "Can Goodness Compete [with power]?" by @jkcarlsmith (link in next post) It's a really fundamental and important question. It's pretty explicit going back at least as far Hammurabi; Plato skirts around it; Jesus is deeply concerned with it; and it's of…
It's hard to plan for AGI without knowing what outcomes are even possible, let alone good. So we’re hosting a workshop! Post-AGI Civilizational Equilibria: Are there any good ones? Vancouver, July 14th Featuring: @jkcarlsmith @RichardMCNgo @eshear 🧵
💡Leading researchers and AI companies have raised the possibility that AI models could soon be sentient. I’m worried that too few people are thinking about this. Let’s change that. I’m excited to announce a Digital Sentience Consortium. Check out these funding opps.👇
To my knowledge, this is the most serious industry-led attempt to investigate the welfare of a frontier AI system in human history. Kudos to Anthropic for leading the way.
🧵For Claude Opus 4, we ran our first pre-launch model welfare assessment. To be clear, we don’t know if Claude has welfare. Or what welfare even is, exactly? 🫠 But, we think this could be important, so we gave it a go. And things got pretty wild…