Disclosure: the essay and this summary were drafted with Claude from my own ideas and stories.
Evan Hubinger put the chance of AI killing everyone at over 10% within a decade, and the largest survey of AI researchers lands on a 10% median. I've spent years managing risk on rock faces, and at 22, at the European Space Agency, worked out a satellite's fuel budget to 99.7% confidence, so I tried judging that number the way engineers judge anything with lives attached. The systems on an airliner that could bring it down are each designed to fail less than once in a billion flight hours. In a 2025 Australian survey, 94% of people said AI should be held to safety standards at least as strict as commercial aviation. Measured against it, 10% isn't close. Interested in where people think the analogy breaks.