State vs. step risk
The cumulative state risk associated with being in some state grows as a function of the time spent in that state. Natural existential risks are typically state risks. For example, absent deflection efforts, the risk that an asteroids of a certain size collides with Earth by 2030 is higher than the risk that it does so by 2029. The longer humanity exposes itself to a state risk, the higher its probability of succumbing to the associated catastrophe. For this reason, thethere are pro tanto reasons for reducing state risks as soon as possible.
Ord, Toby (2020) The Precipice: Existential Risk and the Future of Humanity, London: Bloomsbury Publishing, ch. 7.
Bostrom, Nick (2014) Superintelligence: Paths, Dangers, Strategies, Oxford: Oxford University Press.Press, ch. 14.
A state risk is a risk associated with being in a particular state, whereas a step risk (also called a transition risk) is a risk arising during a transitionfrom transitioning to a new state. Nick Bostrom appears to have originated the distinction in his book Superintelligence, although some mentions of it predate the book's publication.[1]
A state risk is a risk associated with being in a particular state, whereas a step risk (also called a transition risk) is a risk arising during a transition to a new state. Nick Bostrom appears to have originated the distinction in his book Superintelligence, although some mentions of it predate the book's publication (e.g. Beckstead 2013).publication.[1]
Since state risks are correlated with natural existential risks, and step risks with anthropogenic existential risks, the latter's much greater share of total existential risk suggests that most of this risk is posed by transitioning to new states, rather than by remaining in a given state (Ord 2020).state.[2] This finding has important implications for the strategic management of existential risk.
BibliographyFurther reading
Beckstead, Nick (2013) How to compare broad and targeted attempts to shape the far future, July 13.
Ord, Toby (2020) The Precipice: Existential Risk and the Future of Humanity, London: Bloomsbury Publishing.
- ^
For example Beckstead, Nick (2013) How to compare broad and targeted attempts to shape the far future, July 13.
- ^
Ord, Toby (2020) The Precipice: Existential Risk and the Future of Humanity, London: Bloomsbury Publishing.
A state risk is a risk associated with being in a particular state, whereas a step risk (also called a transition risk) is a risk arising during a transition to a new state.Nick Bostrom appears to have originated the distinction in his book Superintelligence, although some mentions of it predate the book's publication (e.g. Beckstead 2013).
The cumulative state risk associated with being in some state grows as a function of the time spent in that state. Natural existential risks are typically state risks. For example, absent deflection efforts, the risk that an asteroids of a certain size collides with Earth by 2030 is higher than the risk that it does so by 2029. The longer humanity exposes itself to a state risk, the higher its probability of succumbing to the associated catastrophe. For this reason, the are pro tanto reasons for reducing state risks as soon as possible.
Things are different with step risks. Here the threat arises only when the transition to the new state begins, and the overall risk incurred during this transition is not generally a function of its total duration. Thus, with step risks there is no presumption in favor or against either postponing or prolonging the transition; what is appropriate will vary depending on characteristics specific to each risk. Some anthropogenic existential risks are plausibly viewed as step risks, with AI risk being perhaps the clearest example.
Since state risks are correlated with natural existential risks, and step risks with anthropogenic existential risks, the latter's much greater share of total existential risk suggests that most of this risk is posed by transitioning to new states, rather than by remaining in a given state (Ord 2020). This finding has important implications for the strategic management of existential risk.
Bibliography
Beckstead, Nick (2013) How to compare broad and targeted attempts to shape the far future, July 13.
Bostrom, Nick (2014) Superintelligence: Paths, Dangers, Strategies, Oxford: Oxford University Press.
Ord, Toby (2020) The Precipice: Existential Risk and the Future of Humanity, London: Bloomsbury Publishing.
Sandberg, Anders (2017) Survivorship curves and existential risk, Andart II, February 8.
Related entries
anthropogenic existential risk | existential risk | natural existential risk | total existential risk
The cumulative state risk associated with being in some state grows as a function of the time spent in that state. Natural existential risks are typically state risks. For example, absent deflection efforts, the risk that an
asteroidsasteroid of a certain size collides with Earth by 2030 is higher than the risk that it does so by 2029. The longer humanity exposes itself to a state risk, the higher its probability of succumbing to the associated catastrophe. For this reason, there are pro tanto reasons for reducing state risks as soon as possible.