David Manheim
@davidmanheim
Lecturer @TechnionLive, founder @alter_org_il, emeritus @superforecaster, PhD @PardeeRAND Optimistic on AI, pessimistic on humanity managing the risks well.
This post says solving the alignment problem is step 1 of the best path forward, but OpenAI appears to be skipping that step and racing to build superintelligence anyways?
Organizations don't act based on individual beliefs, they act based on their structure, precedents, and legal constraints - and in policy, by needing consensus that actions won't be upsetting to anyone nor attract undue attention.
For what it is worth, few industry leaders, less than a half-dozen companies & no policy-making bodies are taking actions that suggest that they expect AGI is really a few years away. This may be because they don’t believe it or they think it won’t matter much in the medium term
labs be like "misalignment is fake and just caused by bad things in the training data", and then not filter out the bad things from the training data
in general i think we have bad conceptualizations of parenting: what it is, what the clusters within it are, what kinds of activities different groups undertake, etc. i'd love to see more work in the area. but i'm kind of skeptical that "parenting isn't fun" is actually a take…
Want to have a more impactful career but don't know where to start? Unsure whether to focus on animal welfare or risks from AI? This week, you can ask career advisors from @80000Hours, @Probably_Good_ , @successif_org and Animal Advocacy Careers anything, via our AMA:
AI proponents keep publicly saying they're surprised by the pace of progress. The past public claims and forecasts are accessible. Do you think they understated the pace of progress years ago so they'd get less hype/investor interest then, and more now? x.com/alexwei_/statu…
9/N Still—this underscores how fast AI has advanced in recent years. In 2021, my PhD advisor @JacobSteinhardt had me forecast AI math progress by July 2025. I predicted 30% on the MATH benchmark (and thought everyone else was too optimistic). Instead, we have IMO gold.
The people building AI keep being surprised by their own progress, and most of them think the systems will be able to replace essentially all human jobs in next couple years, and will pose existential threats if not aligned soon afterwards. And despite that, they are racing…
9/N Still—this underscores how fast AI has advanced in recent years. In 2021, my PhD advisor @JacobSteinhardt had me forecast AI math progress by July 2025. I predicted 30% on the MATH benchmark (and thought everyone else was too optimistic). Instead, we have IMO gold.
I’m giving a talk on the speed of progress on LLM capabilities in 3 hours, gotta update the slides 😭😭
1/N I’m excited to share that our latest @OpenAI experimental reasoning LLM has achieved a longstanding grand challenge in AI: gold medal-level performance on the world’s most prestigious math competition—the International Math Olympiad (IMO).
Today we launched a new product called ChatGPT Agent. Agent represents a new level of capability for AI systems and can accomplish some remarkable, complex tasks for you using its own computer. It combines the spirit of Deep Research and Operator, but is more powerful than that…
Today we launched a new product called ChatGPT Agent. Agent represents a new level of capability for AI systems and can accomplish some remarkable, complex tasks for you using its own computer. It combines the spirit of Deep Research and Operator, but is more powerful than that…
During dinner yesterday I found out that all the others (upper middle class professionals, well-off academics, internationally oriented) were hesitant to even visit the US. These are the people most countries would really want to come and spend money, do business, and contribute.
Trustworthy experts admit uncertainty and admit when there's disagreement. People wanting to sound serious do not. But science as a process works when there are competing ideas and places to ask questions. Faked seriousness and claimed certainty isn't compatible with science!
And the point Rob makes about the problems claiming certainty in these domains is an important one. Because, unfortunately, not admitting you might be wrong, and not saying that you're uncertain, makes you *seem* to have more expertise, and *seem* more serious, to non-experts.
It's about scarcity. That's how supply and demand work. Turns out that there are incredibly few people who have signficant practical experience doing some very specific tasks needed for building frontier models, at the scale needed, which they are being hired for.
There are two theories for why some AI engineers are worth $100mil+ salaries: 1. Some engineers are really so good that they generate that much value 2. The higher you are in an org, the more you know. Even if you do nothing after being hired, what you know is worth $100mil+
After trying this for a bit, it turns out that the algorithmic "for you" timeline really does provide useful tweets that don't show up on my various lists or other tweetdeck columns. Reluctantly recommended.
It in fact shows me interesting things I don't yet get from Following.
Fantastic framing for addressing lots of immediate AI risks, and very valuable work - but I'm very skeptical that this virtuous cycle would enable insurance for the potential existential risks, which @so8res discussed a few years ago: lesswrong.com/posts/mSeesg7i…
Here's the gist: Insurers have incentives and power to enforce that the companies they insure take action to prevent the risks that matter. They enforce security through an incentive flywheel: Insurers create standards. Standards outline which risks matter and what companies…
Counter-proposal: Mandatory national service for people *when they turn 65*. They have expertise, experience, and the ability to give so much back to their communities and their country! nytimes.com/2019/06/21/mag…
I think I figured out what happened: Elon asked @xAI to make a new SOTA LLM, and evidently one of the engineering teams somehow thought that meant Sexy Obsequious Toxic Anime-bot.