"Military AI as a Convergent Goal of Self-Improving AI"
.
Forthcoming as a chapter in Artificial Safety And Security (Roman V. Yampolskiy, ed.), CRC Press.
Abstract
Better instruments to predict the future evolution of artificial intelligence (AI) are needed, as the destiny of our civilization depends on it. One of the ways to such prediction is the analysis of the convergent drives of any future AI, started by Omohundro. We show that one of the convergent drives of AI is a militarization drive, arising from AI’s need to wage a war against its potential rivals by either physical or software means, or to increase its bargaining power. This militarization trend increases global catastrophic risk or even existential risk during AI takeoff, which includes the use of nuclear weapons against rival AIs, blackmail by the threat of creating a global catastrophe, and the consequences of a war between two AIs. As a result, even benevolent AI may evolve into potentially dangerous military AI. The type and intensity of militarization drive depend on the relative speed of the AI takeoff and the number of potential rivals. We show that AI militarization drive and evolution of national defense will merge, as a superintelligence created in the defense environment will have quicker takeoff speeds, but a distorted value system. We conclude with peaceful alternatives.
Link without registration and opened for commenting:
This month effective altruism has gotten lots of attention, including a cover story in The Economist. It feels pretty surreal to be on the receiving end of it. But I feel grateful these ideas have caught on, and even more grateful for the drive, earnestness, and compassion of so many people who took strange but well-argued ideas seriously and then actually acted on them....
We’re the co-founders of Better Futures Guide (BFG), a charity evaluator specifically focused on livelihoods. We were founded last year through Ambitious Impact’s Charity Entrepreneurship Incubation Program. We welcome questions, criticism and opportunities for collaboration, in the comments or at info@betterfuturesguide.org.
...
TL;DR
* We recently set out RP’s strategic priorities and the three new bets we’re making in 2027.
* In this post, we outline what we need to make them happen.
* We need more support from individual donors, this year in particular.
* We’ve redirected roughly three-quarters of our 2026 unrestrict...