The Moral Blind Spot in AI Governance: Risks of Large-Scale Suffering

We’ve just released a paper with the above title where we argue that AI safety discussions warrant much greater attention to suffering caused by misused or misaligned AI, as well as from an entire category of overlooked scenarios: uncontrolled AI aligned with values that don’t prioritize suffering prevention. We write, "Technically aligning powerful AI systems with our current, flawed societal values and practices, heterogeneous as they are, would not amount to genuine moral alignment. It would risk preserving and amplifying our present moral blind spots and even locking them in as durable features of the future." We include the abstract and table below.

Abstract

Large-scale intense suffering resulting from the deployment of AI systems represents a moral blind spot that AI governance has largely overlooked. The underlying issue is not primarily technical but one of values: a failure of societal norms and practices to prioritize suffering prevention. We highlight three categories of AI-driven suffering risks: 1. Human misuse of AI; 2. Uncontrolled misaligned AI; and, in particular, 3. Uncontrolled AI aligned with flawed values. We urge policymakers and developers to: 1. Prioritize suffering prevention in AI governance and development; 2. Assess AI systems’ impacts on suffering; 3. Ensure responsible human control over AI systems; and 4. Pursue international cooperation centered on compassionate values.

Table