iYears until artificial superintelligence (ASI) arrives if development is paused or heavily restricted. A pause delays unaligned ASI — but it also delays the aligned ASI that could help solve the other existential risks below. More labs in the race shortens this timeline.❌ Pause Scenario
19 years
iYears until ASI arrives with open, unrestricted development. Arriving sooner means aligned ASI can start reducing the other existential risks sooner — if alignment succeeds. More labs in the race shortens this timeline.✅ Open Scenario
8 years
🤖 Unaligned ASI risk (multi-lab model)
iWith all labs being secret, no lab benefits from another lab’s alignment breakthroughs — every lab must independently get alignment right. So each additional lab adds to the risk of unaligned ASI. p(doom) is probability of doom.❌ Pause (secret labs — no sharing — all must get it right)
38%
12 labs
iOpen labs will tell the world how to make aligned ASI if they succeed, so adding more open labs doesn’t increase unaligned ASI risk — it improves the odds that someone solves alignment first. Secret labs don’t share, so they still add risk. p(doom) is probability of doom.✅ Open (just one open lab must get it right for all to win)
8%
24 labs
6 labs
☢️ Other Existential RisksiNick Bostrom’s Vulnerable World Hypothesis pictures invention as drawing balls from an urn — most white (beneficial), a few black (civilization-ending by default). Aligned ASI is the ultimate white ball: the one technology that could spot and stop the black balls below before they are drawn. Pausing ASI doesn’t stop the drawing — it just leaves humanity facing these risks without its best defense.
iIf aligned ASI arrives before the doom avoidance point, nuclear risk drops to the “with ASI” value; otherwise the “without ASI” value applies.
The last slider covers enforcement risk: Yudkowsky has called for an AI pause enforced by airstrikes on data centers in countries exceeding compute limits that might indicate AI training runs — similar to the strikes used against Iran to deter nuclear weapons. But some non-complying countries already have nuclear weapons, so enforcement itself adds nuclear risk. This extra risk applies only in the Pause scenario.🔥 Nuclear War
12 years
19%
3%
12%
iSelf-replicating nanotechnology could consume the biosphere (“gray goo”). If aligned ASI arrives before the doom avoidance point, it can help develop defenses and safe replication protocols, dropping the risk to the “with ASI” value; otherwise the “without ASI” value applies. Dramatized as Doom 2 (Crimson Eternal) in Chapter 6 of the book.🧬 Nanotech (gray goo)
18 years
15%
1%
iFuture particle physics experiments could theoretically trigger a catastrophe such as vacuum decay or stable strangelets. If aligned ASI arrives before the doom avoidance point, it can verify experiment safety in advance, dropping the risk to the “with ASI” value; otherwise the “without ASI” value applies. Dramatized as Doom 3 (A Big Bang) in Chapter 6 of the book.⚛️ High Energy Physics
22 years
9%
3%
iEngineered pathogens — including viruses targeted at specific populations — become more dangerous as biotech gets cheaper and more accessible. If aligned ASI arrives before the doom avoidance point, it can provide rapid detection, vaccines, and countermeasures, dropping the risk to the “with ASI” value; otherwise the “without ASI” value applies. Dramatized as Doom 1 (the Bullseye Virus) in Chapter 6 of the book.🦠 Biotech (targeted viruses)
11 years
24%
2%
🌍 OVERALL EXISTENTIAL RISK
iCombines all five risk categories by multiplying their survival probabilities together — you must survive every one of them — then total p(doom) = 100% minus that combined survival. This is why the total is less than the sum of the breakdown numbers: multiplying avoids double-counting worlds where two dooms would both occur.
Unaligned ASI risk compounds across secret labs, since every lab must independently get alignment right. And because a pause delays ASI, aligned ASI usually arrives too late to help with the other risks, so their higher “without ASI” probabilities apply — plus the extra nuclear risk from enforcement airstrikes.❌ Pause Scenario — Total p(doom)
…
Most likely doom:…
iCombines all five risk categories by multiplying their survival probabilities together — you must survive every one of them — then total p(doom) = 100% minus that combined survival. This is why the total is less than the sum of the breakdown numbers: multiplying avoids double-counting worlds where two dooms would both occur.
Unaligned ASI risk stays low if any open lab solves alignment before a secret lab causes doom. And because ASI arrives sooner, it often beats the doom avoidance points for the other risks, dropping them to their lower “with ASI” probabilities.✅ Open Scenario — Total p(doom)
…
Most likely doom:…
Pause Scenario ASI risk: all labs are secret and none share, so every lab must independently get alignment right → p(doom) = 1 - (1-p)^n_labs.
Open Scenario ASI risk (with secret labs):
• 0 labs total → p(doom) = 0
• Only secret labs → p(doom) = 1 - (1-p)^n_secret
• Open + secret labs → exact combinatorics: probability that all secret labs before the first open lab succeed + the first open lab succeeds.
Other risks: if the effective ASI arrival time beats a risk’s doom avoidance point, its “with ASI” probability applies; otherwise its “without ASI” probability applies. In the Pause Scenario, doom avoidance points arrive 15% sooner (a pause slows defensive technology too), and the extra nuclear risk from enforcement airstrikes is added.
Total p(doom) = 100% minus the product of all five categories’ survival probabilities.