An existential risk is the risk of an existential catastrophe, i.e. one that threatens the destruction of humanity’s longterm potential.^[1]^[2] Existential risks include natural risks such as those posed by asteroids or supervolcanoes as well as anthropogenic risks like mishaps resulting from synthetic biology or artificial intelligence.

A number of authors have argued that existential risks are especially important because the long-run future of humanity matters a great deal.^[1]^[3]^[4]^[5] Many believe that there is no intrinsic moral difference between the importance of a life today and one in a hundred years. However, there may be many more people in the future than there are now. They argue, therefore, that it is overwhelmingly important to preserve that potential, even if the risks to humanity are small.

One objection to this argument is that people have a special responsibility to other people currently alive that they do not have to people who have not yet been born.^[6] Another objection is that, although it would in principle be important to manage, the risks are currently so unlikely and poorly understood that existential risk reduction is less cost-effective than work on other promising areas.

Recommendations

In The Precipice: Existential Risk and the Future of Humanity, Toby Ord offers several policy and research recommendations for handling existential risks:^[7]

Explore options for new international institutions aimed at reducing existential risk, both incremental and revolutionary.
Investigate possibilities for making the deliberate or reckless imposition of human extinction risk an international crime.
Investigate possibilities for bringing the representation of future generations into national and international democratic institutions.
Each major world power should have an appointed senior government position responsible for registering and responding to existential risks that can be realistically foreseen in the next 20 years.
Find the major existential risk factors and security factors—both in terms of absolute size and in the cost-effectiveness of marginal changes.
Target efforts at reducing the likelihood of military conflicts between the US, Russia and China.
Improve horizon-scanning for unforeseen and emerging risks.
Investigate food substitutes in case of extreme and lasting reduction in the world’s ability to supply food.
Deve

...

Credo AI