Peter Barnett
@peterbarnett_
Trying to ensure the future is bright. Researcher at @MIRIBerkeley Views my own.
5 Billion, But Not A Single One More
they fucking did it they genetically edited mosquitoes so they can’t transmit malaria. and made the mutation self-replicating. edited mosquitoes and every descendant can’t infect humans.
The AI Action Plan says we must ensure that AI procured by the government “objectively reflects truth”. This means that they will need a definition of “truth” in order to set policy. Luckily I know of some relevant experts
Risk of lice goes up for EDT reasons (you're more likely to have lice in worlds where you check for lice) (This is a joke, it actually goes down because checking for lice is cooperative, and so due to acausal trade reasons you're more likely to be simulated without lice)
A preschool (this is real) requires that all students get checked for lice before coming the first day. You do this by taking them to a place people check kids for lice. Does this make the risk of lice on that first day:
labs be like "misalignment is fake and just caused by bad things in the training data", and then not filter out the bad things from the training data
So true
Strong agree w/ this from @jackclarkSF "Right now, society does not have the ability to choose to stop the creation of a superintelligence if it wanted to. That seems bad!"
We’ll be presenting two posters at the @taig_icml workshop this Saturday! Come say hi, and learn about: - Technical interventions needed to globally shut down dangerous AI activities - How much compute is currently used for AI algorithmic progress
I feel a bit sad when people (both inside and outside of the AI x-risk sphere) don’t cite/reference some of the original work on a topic because they think that the author is uncool/taboo. Feels a bit low integrity.
"I’m worried AI will get to the point where someone can kick of recursive self improvement with just their web browser." "I don’t quite understand, can you say that in a different way?" "i.e., IE IE."
As Zach Stein-Perlman says, it is bad that xAI hasn't published (or even performed?) any dangerous capability evals for Grok 4. This is much worse than other AI companies like OpenAI, GDM and Anthropic.
Crazy that the OpenAI "Our Structure" page openly miss-states OpenAI's mission in the first sentence. The mission is to ensure AGI benefits humanity, this does *not* necessarily mean building AGI.
We're running a design contest for advertisements for our forthcoming book, "If Anyone Builds It, Everyone Dies". 4x $1000 main prizes; $200 for anything else useful; deadline Aug 10. (Link below.)
I'll be in DC until Sunday, let me know if you want to meet up! I'll be at the AI Security Forum on Wednesday. Excited to talk about AI governance plans and priorities, focused on taking ASI seriously.
Huh this eval seems to be able to capture that there is something different about Opus 3
Opus Was Different
This prediction was wrong (🎉) x.com/METR_Evals/sta…
Weak prediction that Claude 4 will be closer to the AI 2027 extrapolation than the METR trend (including the METR trend updated for o3).
Based on the latest EA vegan discourse, I’m going to be replacing honey with oysters in all my cooking from now on