The ethics of existential risk concerns the questions of how bad an existential catastrophe would be, how good it is to reduce existential risk, precisely why those things are as bad or good as they are, and how this differs between different specific existential risks. There are a range of different perspectives on these questions, and these questions have implications for how much to prioritise reducing existential risk and which specific risks to prioritise reducing.

In The Precipice, Toby Ord discusses five different "moral foundations" for assessing the value of existential risk reduction, depending on whether emphasis is placed on the future, the present, the past, civilizational virtues or cosmic significance.^[1]

The future

In one of the earliest discussions of the topic, Derek Parfit offers the following thought experiment:^[2]

I believe that if we destroy mankind, as we now can, this outcome will be much worse than most people think. Compare three outcomes:
Peace.
A nuclear war that kills 99% of the world's existing population.
A nuclear war that kills 100%.
(2) would be worse than (1), and (3) would be worse than (2). Which is the greater of these two differences? Most people believe that the greater difference is between (1) and (2). I believe that the difference between (2) and (3) is very much greater.

The scale of what is lost in an existential catastrophe is determined by humanity's long-term potential—all the value that would be realized if our species survived indefinitely. The universe's resources could sustain a total of around $10^{35}$ biological human beings, or around $10^{58}$ digital human minds.^[3] And this may not exhaust all the relevant potential, if value supervenes on other things besides human or sentient minds, as some moral theories hold.

In the effective altruism community, this is probably the ethical perspective most associated with existential risk reduction: existential risks are often seen as a pressing problem because of the astronomical amounts of value or disvalue potentially at stake over the course of the long-term future.

The present

Some philosophers have defended views on which future or contingent people do not matter morally.^[4] Even on such views, however, an existential catastrophe could be among the worst things imaginable: it would cut short the lives of every living moral patient, destroying...

ALLFED

Further reading

External links

Related entries