Paul gives some numbers on AI doom (text below). Here they are in graphical forms, which I find easier to understand. Please correct me if wrong.
Michael Trazzi's Probability Flow Diagram
I really like this one. I can really easily read how he thinks future worlds are distributed. I guess the specific flows are guesses from Paul's model so might be wrong but I think it's fine.
My probability model version
This is messier, but interactive. You get to see what the chances Paul puts on specific breakpoints are. Do you disagree with any?
Paul's model in text
Probability of an AI takeover: 22%
- Probability that humans build AI systems that take over: 15%
(Including anything that happens before human cognitive labor is basically obsolete.)
- Probability that the AI we build doesn’t take over, but that it builds even smarter AI and there is a takeover some day further down the line: 7%
Probability that most humans die within 10 years of building powerful AI (powerful enough to make human labor obsolete): 20%
- Probability that most humans die because of an AI takeover: 11%
- Probability that most humans die for non-takeover reasons (e.g. more destructive war or terrorism) either as a direct consequence of building AI or during a period of rapid change shortly thereafter: 9%
Probability that humanity has somehow irreversibly messed up our future within 10 years of building powerful AI: 46%
- Probability of AI takeover: 22% (see above)
- Additional extinction probability: 9% (see above)
- Probability of messing it up in some other way during a period of accelerated technological change (e.g. driving ourselves crazy, creating a permanent dystopia, making unwise commitments…): 15%