Hi! My name is Joris, I'm a university group organizer at PISE (the group at Erasmus University Rotterdam, the Netherlands), and I’m currently writing my thesis for my bachelor’s degree in PPE (Philosophy, Politics & Economics). I'm helping Rumtin from GCR Policy study practical policy change on x-risks and GCRs, and concretely I'm exploring the conditions and characteristics of successful policy change for extreme risks.
Policy change on low-likelihood-high-impact risks is challenging to achieve, so finding and studying cases where it has been achieved can help inform the work of the x-risk studies field. The purpose of this research is to find patterns across case studies that can tell us how extreme risk policy occurs. By looking at the key characteristics of past policy efforts, we hope to provide insights on how policy for extreme risks occurs, and to provide guidance to researchers if they wish to do policy engagement.
I have found a number of case studies from which to draw lessons, but I want to broaden my search. I'm reaching out here to ask the following: what would be useful cases of extreme risk policy to study? These case studies should be for policies that were implemented to understand, govern, or reduce extreme risks (not simply existential or globally catastrophic risks). For example, case studies could include: NASA’s asteroid tracking program; US Defense Department’s UFO detection program; closing the hole in the ozone layer; and the joint US-Soviet smallpox and polio vaccination program[1].
We want to look at some other examples (risk- and country-agnostic) to identify common factors that led to policy change, such as individual policy leadership, precipitating events, or other shifts in societal or cultural landscape. Our hypothesis is that precipitating events, particularly small-scale versions of the risk, are the critical factor in shaping policy in that area. For example, a quick review of the literature on NASA's asteroid tracking program identifies a number of drivers for its establishment, including: Comet Shoemaker–Levy 9 hitting Jupiter in 1994, scientific consensus forming around an asteroid causing the dinosaur extinction (including with the discovery of the Chicxulub crater)[2], the releases of blockbusters Deep Impact and Armageddon[3], and the efforts of individual linked to Congress[4]
If you know of any significant extreme risk-related policy successes, we would love to hear. Bonus points if you know of literature (or people) that can help us understand why and how this policy came about!
If you're not comfortable commenting, feel free to email me at joris [dot] pijpers [at] gmail [dot] com!
- ^
https://journals.plos.org/plosntds/article?id=10.1371/journal.pntd.0005320
- ^
See The Precipice, p. 69-70
- ^
Carl Schulman talks about this in this 80.000 Hours podcast episode: https://80000hours.org/podcast/episodes/carl-shulman-common-sense-case-existential-risks/
- ^
See https://www.boulder.swri.edu/clark/ncarhist.html
If you want to draw useful lessons for successful risk governance from this research, it also seems pretty important to collect negative examples of the same reference class, i.e. conditions of extreme risk where policies were proposed but not enacted/enforced, or not proposed at all. E.g. (in the spirit of your example of the DoD's UFO detection program), I don't know of policy governing the risk from SETI-style attempts to contact intelligent aliens.
Are you interested only in public policies related to extreme risk, or examples from corporate governance as well? Corporate risk governance likely happens in a way that's meaningfully different from public policy, and might be relevant for applying this research to e.g. AI labs.
Thanks Michael, also for the suggestions you made above! You raise good points and I would've loved to study negative examples and examples from corporate governance, but the scope of my thesis unfortunately has to be really limited - hopefully someone else can look at these later!
I'm not exactly sure what you have in mind for the research, but I think it might be interesting to at least draw parallels or have pseudo-benchmarks with policy responses to non-existential low-probability risks, such as 9/11 (or terrorism more generally) and US mass shootings.
Thanks Harrison, we're indeed looking at exactly those "policy responses to non-existential low-probability risks" as there is little material out there on policy change regarding GC & X-risks. By 'lowering the bar' a bit to what we called 'extreme risks', we hope to include smaller, less deadly risks into our case study candidates. As such, 9/11 is indeed one to consider, thank!