Philip E. Tetlock
@PTetlock
Penn-Integrates-Knowledge (PIK) Professor, Wharton & School of Arts & Sciences. Likes = interesting; Retweets = very interesting; Interesting ≠ endorsement
Ideological immune systems in overdrive. Cory Clark documents the lengths to which some will go to minimize cognitive dissonance
Five strategies people use to dismiss scientific claims that offend them: 1. Motivated Confusion 2. Motivated Postmodernism 3. The CIA’s Strategies for Citizen-Saboteurs 4. Schopenhauer’s Stratagems for Always Being Right 5. Contradictory Criticisms [Link below.]
Have you ever wondered how far off we are from a world in which, say, biochemistry undergrads can—with AI guidance – “cook up” new lethal subtypes of influenza in virology labs? The FRI study below lays out promising methodology for answering such questions—& the answer is we’re…
@Research_FRI releases new study on risks of bio-engineered epidemics. forecastingresearch.org/ai-enabled-bio… Subject-matter specialists & "superforecaster" generalists expected risks to rise substantially if LLMs passed key virology-lab competency tests. They also thought LLMs would not attain…
FRI found that superforecasters and bio experts dramatically underestimated AI progress in virology: they often predicted it would take 5-10 years for AI to match experts on a benchmark for troubleshooting virology (VCT), but actually AIs had already reached this level.
Our new study finds: recent AI capabilities could increase the risk of a human-caused epidemic by 2-5x, according to 46 biosecurity experts and 22 top forecasters. One critical AI threshold that most experts said wouldn't be hit until 2030 was actually crossed in early 2025. But…
This summary covers the key findings — the full study includes detailed methodology, participant rationales, accuracy measures, and policy implications. Read the complete report from @Research_FRI: forecastingresearch.org/ai-enabled-bio…
Authors: @bridgetw_au, Josh Rosenberg, @lucafrighetti, Rebecca Ceppas de Castro, Otto Kuusela, Rhiannon Britt, Seth Donoughe, Alvaro Morales, Emily Soice, Jon Sanders, James Black and @PTetlock Special acknowledgment to @SecureBio for the virology evaluation collaboration.
Our new study finds: recent AI capabilities could increase the risk of a human-caused epidemic by 2-5x, according to 46 biosecurity experts and 22 top forecasters. One critical AI threshold that most experts said wouldn't be hit until 2030 was actually crossed in early 2025. But…
Solid advice if you are Patrick Collison (or just a top-2% “superforecaster”). But a bad bet for the rest of us. Most of the time, the average judgment of the crowd beats the judgment of individual members of that same crowd. Not inspirational but demonstrably true.
"A large fraction of what people around you believe is mistaken. Internalize this and practice coming up with your own worldview." — Patrick Collison
Social scientists today do their work in silos, surrounded only by those who share their assumptions and intuitions. Adversarial collaborations, therefore, may play a key role in stimulating our collective creativity and scientific innovation. New in Theory and Society:…
Reasonable complaint
Not much in forecasting annoys me more than "this was very predictable" from people who did not make any predictions about it before it happened. I think you'll find that when you try to predict things *before* things happen, you're wrong a lot! It's easy to predict the past!
Hindsight bias fuels forecaster over-confidence. The cure: Remind & re-remind yourself of the deeply incorrect beliefs you once held about the future before you came to hold your current clearly correct beliefs about the future
"IBM is a reliable blue chip stock and Watson is a clear leader in the AI race"
Taboo trade-offs are sources of endless mischief: An inexhaustible supply of sand to throw into the logical machinery of cost-benefit analysis dropbox.com/scl/fi/filekmh…
Appreciate this coverage of gene drives but some of the points here are wild... crazy to focus on the "inherent value" of 1 out of 3,500 species of mosquitoes when almost half a million children die each year from malaria. What about their inherent value? x.com/albrgr/status/…
a great gist packs as much insight into as few words as possible. This is one of the greatest!
"We have created a Star Wars civilization, with Stone Age emotions, medieval institutions and god-like technology." –E. O. Wilson For more great E. O. Wilson quotes, check out the link below.
Really enjoyed working on this with @SchoeneggerPhil @PTetlock and Barbara Mellers. We tried ~50 prompt techniques (including AI classics and more theory-motivated ones) on 100 forecasting questions across 6 LLMs. No prompt showed robust improvements! arxiv.org/abs/2506.01578
New preprint with @CamrobJones @PTetlock and Mellers! We test how much prompt engineering can impact LLM forecasting capabilities of o1, o1-mini, 4o, Sonnet, Haiku & Llama, finding that simple-to-moderate prompt engineering has little or no effect, with some prompts backfiring!
Handle with care: A powerful justification for procrastination
The Paradox of Distance
Useful to measure scientific impact on log scales but not cognitive ability, which is roughly normally distributed and can arguably only be measured on ordinal scales. Makes zero sense to say Einstein was 10X smarter than Bohr.
On Landau's ranking of theoretical physicists. Lev Landau, a Nobelist and one of the fathers of the great Soviet physics system, had a logarithmic scale for ranking theorists, from 1 to 5. A physicist in the first class had ten times the impact of someone in the second class,…
count me as curious too!
Has someone come up with a great prompt for socratic tutoring? Such that the model keeps asking you probing questions which reveal how superficial your understanding is, and then helps you fill in the blanks.
An early 20th century truism that holds true in the early 21st—but that many memory-holed during the dark-age decades when labeling an argument “neo-positivist” sufficed as a refutation
"good sociological theories must be compatible with good psychological theories, which in turn must be compatible with good biological theories, which must be compatible with good chemical theories, which must be compatible with good theory from physics" link.springer.com/article/10.100…
has anyone stopped to ask WHY students cheat? would a buddhist monk "cheat" at meditation? would an artist "cheat" at painting? no. when process and outcomes are aligned, there's no incentive to cheat. so what's happening differently at colleges? the answer is in the article:
Tragedy can create a deeply teachable moment. And we compound tragedy when we refuse to learn because the "enemy" is gloating over our mistakes. So simple; so hard.
FYI: it’s not misogyny or sexist to blame this female pilot or DEI. There are amazing female pilots. But when standards are lowered with identity politics, and trainers are afraid to be harsh on candidates of gender or color, it’s dangerous! Merit saves lives; DEI is deadly.