Hide table of contents

I've been worried about x-risk for a very long time. But for some reason AI x-risk has never felt intuitive to me.

On an intellectual level AI risk makes sense to me, but I would like to be able to visualize what a few plausible AI take-over scenarios could concretely look like. I would really appreciate if anyone could share books and movies with realistic AI elements and great storytelling that helped you to connect emotionally to the scariness of AI x-risk. 

Alternatively, I would appreciate recommendations for any content, posts, ideas, revelations, experiences, "a-ha" or "0h-shit" moments, etc. - anything that helped you realize how seriously scary AI is and motivated you to work on it.

Thank you for the advice!

11

0
0

Reactions

0
0
New Answer
New Comment


8 Answers sorted by

Ex Machina does a great job of demonstrating the AI boxing experiment.

My favorite is probably the movie Colossus: the Forbin Project. For this, would also weakly recommend the first section of Life 3.0. 

I think the series Next did a pretty good job of making me scared. It's not an amazing production in itself, but worth watching.

love the answers so far. will also toss in Mission: Impossible - Dead Reckoning (allegedly partially inspired Biden's EO), Her (almost a decade before character.ai), Westworld, and Moon

Besides the typical (over-dramatised) killer robot scenarios, I would add the perspective of looking for infrastructure breakdowns or societal chaos. Imagine a book like Blackout with a disruptive, national blackout but that was caused by powerful intelligent (control) systems. 
Or a movie like Don't Look Up where AI is used to or actively spreads misinformation that severely impact public opinion and taking effective action. 
In movies and books it might commonly be portrayed (in part) as human failure; but it could have been the result of correlated automation failure or goal mis-specification or a power-seeking AI system. 

The consequences that individual humans and society at large suffer from critical infrastructure breaking down can be quite realistic and visceral. 

Some examples not mentioned already, (and of course YMMV for each):

That Alien Message by Yudkowsky is very short, and might get at the intuition

It Looks Like You’re Trying To Take Over The World by Gwern is full of annonated links, if you want to get very technical and concrete about things

For a full-length book, I think that the Crystal Trilogy by Max Harms does a very good job, and would recommend if you like similar sci-fi

Finally, I would actually recommend the film Upgrade. While nowhere near as good a film as Ex-Machina, it's a good illustration that the AI risk is about malicious software agents, and not "evil robots" (i.e. Terminator) which I still think is the dominant public image when prompted about AI risk. Also, really cool fight choreography.

What this post and answer shows me… is that we probably need more material? If we want to ‘scare people’ about AI and make more people aware of it as an issue.

We’ve got lots of great (and dense) YouTube and blog content on the technical aspects. We could do with more more “Holy Shit Imagine an AI That Does This!!” type-content.

For me, the easiest to imagine model of how an AI takeover could look like has been depicted in Black Mirror: Shut Up and Dance (the episodes are fully independent stories). It's probably just meant to show scary things humans can do with current technology, but such schemes could be trivial for a superintelligence with future technology.

Curated and popular this week
trammell
 ·  · 25m read
 · 
Introduction When a system is made safer, its users may be willing to offset at least some of the safety improvement by using it more dangerously. A seminal example is that, according to Peltzman (1975), drivers largely compensated for improvements in car safety at the time by driving more dangerously. The phenomenon in general is therefore sometimes known as the “Peltzman Effect”, though it is more often known as “risk compensation”.[1] One domain in which risk compensation has been studied relatively carefully is NASCAR (Sobel and Nesbit, 2007; Pope and Tollison, 2010), where, apparently, the evidence for a large compensation effect is especially strong.[2] In principle, more dangerous usage can partially, fully, or more than fully offset the extent to which the system has been made safer holding usage fixed. Making a system safer thus has an ambiguous effect on the probability of an accident, after its users change their behavior. There’s no reason why risk compensation shouldn’t apply in the existential risk domain, and we arguably have examples in which it has. For example, reinforcement learning from human feedback (RLHF) makes AI more reliable, all else equal; so it may be making some AI labs comfortable releasing more capable, and so maybe more dangerous, models than they would release otherwise.[3] Yet risk compensation per se appears to have gotten relatively little formal, public attention in the existential risk community so far. There has been informal discussion of the issue: e.g. risk compensation in the AI risk domain is discussed by Guest et al. (2023), who call it “the dangerous valley problem”. There is also a cluster of papers and works in progress by Robert Trager, Allan Dafoe, Nick Emery-Xu, Mckay Jensen, and others, including these two and some not yet public but largely summarized here, exploring the issue formally in models with multiple competing firms. In a sense what they do goes well beyond this post, but as far as I’m aware none of t
Rasool
 ·  · 1m read
 · 
In 2023[1] GiveWell raised $355 million - $100 million from Open Philanthropy, and $255 million from other donors. In their post on 10th April 2023, GiveWell forecast the amount they expected to raise in 2023, albeit with wide confidence intervals, and stated that their 10th percentile estimate for total funds raised was $416 million, and 10th percentile estimate for funds raised outside of Open Philanthropy was $260 million.  10th percentile estimateMedian estimateAmount raisedTotal$416 million$581 million$355 millionExcluding Open Philanthropy$260 million$330 million$255 million Regarding Open Philanthropy, the April 2023 post states that they "tentatively plans to give $250 million in 2023", however Open Philanthropy gave a grant of $300 million to cover 2023-2025, to be split however GiveWell saw fit, and it used $100 million of that grant in 2023. However for other donors I'm not sure what caused the missed estimate Credit to 'Arnold' on GiveWell's December 2024 Open Thread for bringing this to my attention   1. ^ 1st February 2023 - 31st January 2024
 ·  · 1m read
 · 
[This post was written quickly and presents the idea in broad strokes. I hope it prompts more nuanced and detailed discussions in the future.] In recent years, many in the Effective Altruism community have shifted to working on AI risks, reflecting the growing consensus that AI will profoundly shape our future.  In response to this significant shift, there have been efforts to preserve a "principles-first EA" approach, or to give special thought into how to support non-AI causes. This has often led to discussions being framed around "AI Safety vs. everything else". And it feels like the community is somewhat divided along the following lines: 1. Those working on AI Safety, because they believe that transformative AI is coming. 2. Those focusing on other causes, implicitly acting as if transformative AI is not coming.[1] Instead of framing priorities this way, I believe it would be valuable for more people to adopt a mindset that assumes transformative AI is likely coming and asks: What should we work on in light of that? If we accept that AI is likely to reshape the world over the next 10–15 years, this realisation will have major implications for all cause areas. But just to start, we should strongly ask ourselves: "Are current GHW & animal welfare projects robust to a future in which AI transforms economies, governance, and global systems?" If they aren't, they are unlikely to be the best use of resources. 
Importantly, this isn't an argument that everyone should work on AI Safety. It's an argument that all cause areas need to integrate the implications of transformative AI into their theory of change and strategic frameworks. To ignore these changes is to risk misallocating resources and pursuing projects that won't stand the test of time. 1. ^ Important to note: Many people believe that AI will be transformative, but choose not to work on it due to factors such as (perceived) lack of personal fit or opportunity, personal circumstances, or other pra