I'm sympathetic to AI-risk being a top cause area, but I find it hard to fully grasp the urgency because I struggle to picture what an AI catastrophe could actually look like.
From conversations I’ve had with colleagues, friends, and family (including other EAs), I think the lack of vivid, concrete examples is a major barrier for many people caring about this more. I’d love to hear examples of how an AI-related catastrophe might unfold (~500 words each). Examples should describe a powerful, but somewhat limited AI to make the example seem more realistic to an audience that is already skeptical of AGI. Specifically:
- Nominal use-case: What is the AI supposed to be used for? Why did people think it was worth the cost/benefit to release?
- Misalignment: What was the misalignment, and what harmful behavior resulted?
- Logistics: How exactly did the AI interact with the physical world to harm/kill people? Why were humans unable to stop the AI from hurting/killing people once people realized this was happening?
- Difficulty foreseeing: What factors could've made it difficult for even thoughtful designers to anticipate the failure?
These examples should feel vivid enough that someone could easily share them with a less technical and less EA-aligned audience. Ideally, a journalist could write/describe them in a mainstream media source like The New York Times , Vox (@Kelsey Piper - you're an awesome writer!), or Fox News.
Maybe examples like this already exist, and I’ve missed them. If so, please point me in the right direction! If not, I think developing and sharing such scenarios could make the risks feel much more tangible and help motivate more people to care about the issue.
For what it's worth, I think the following are great resources, but don't provide sufficiently succinct, tangible, or compelling examples of AI-related catastrophes for a general audience: Future of Life, 2024, Center for AI Safety, 2023, 80,000 hours, 2022, Vox, 2020.
Photo from CNAS.