Hide table of contents

Summary/TLDR: The EA Community exerts almost no focus on state sovereignty as a crucial consideration in mitigating existential risks. Efforts to mitigate existential risks in tandem with the international community have broadly restricted themselves to technical problem-solving, incentive-building and capacity-building. This leaves out a considerable set of cases where states are non-compliant with actions that could improve existential security. As a relative newcomer to the community who has an academic background in international relations and international law - accompanied by personal experiences as a political activist agitating against authoritarian governments - I find this to be an incredibly obvious and considerable problem, that has been relatively neglected by the community. 

Simply put, I believe that many x-risk reduction strategies are completely unworkable without some restriction of state sovereignty. 

Here I make the case for it becoming a fundamental part of global priorities research, investigate possible critiques of this strategy and posit a set of recommendations. 

Note: This post is both intended to be a submission for the EA Criticism and Red-Teaming Contest, as well as an edited excerpt of the first draft of a research paper[1] titled "Reimagining 'Epidemic Sovereignty': Reconceptualizing Governance of Global Catastrophic Biological Risks". Feedback is, hence, very welcome. 


Existential risk reduction is often conceptualized as a global, transgenerational public good. It is instructive, however, to consider what unit of analysis best fits this frame, i.e. who exactly are they public goods for? The referrant political object here is often 'humans', and while this may be the correct ethical choice, it is equally valuable to think of it as states. 

The question of what qualifies as a state is oddly contentious (until one thinks of ISIS claiming a state, or the claims for self-determination that emenate from Kashmir, northern Sri Lanka, Scotland, Catalonia and many other territories). Modern discussion on it stems from a set of criteria established (and broadly agreed upon) at the Montevideo Convention which defines a state as having: 

  1. A settled population,
  2. A defined territory
  3. An established government and,
  4. The ability to enter into relations with other states

The defining feature of a state is that it has sovereignty, which implies a hierarchy of authority within the state (domestic sovereignty) and autonomy/freedom from interference of other states (external sovereignty). Stephan Krasner, perhaps the most well-known contemporary theorist of sovereignty, posits four categories of sovereignty, all of which must exist for a state to be sovereign: 

  • Domestic sovereignty – actual control over a state exercised by an authority organized within this state,
  • Interdependence sovereignty – actual control of movement across state's borders
  • International legal sovereignty – formal recognition by other sovereign states
  • Westphalian sovereignty – there is no other authority in the state aside from the domestic sovereign.

All of these categories rest on a common assumption - that sovereignty must be recognized for it to have any meaning. This leads on to another fundamental distinction between de jure and de facto sovereignty. De jure sovereignty refers to an institutionally recognized right to exercise authority over a territory free from interference, while de facto sovereignty descibes a situation where that is the basic reality even if it is not institutionally recognized. So, for example, Russia has de facto sovereign control over Crimea even if it does not have de jure sovereign control over the territory. 

 This post is interested in external sovereignty in both de jure and de facto terms. It claims that sovereignty - insofar as it is freedom from external interference - can be a legitimate organizing principle on which to premise global relations, but creates incredibly large difficulties for pushing for mitigation of risks which are inherently global. 

Even if the EA community can devise robust remedies which mitigate x-risks, the success of many of these are reliant on some level on global enforcement, which is constrained by state sovereignty. 

As Table 1 (below) demonstrates, x-risk mitigation policies are reliant on enforcement, which it itself reliant on authority, and the main organizing principle of authority in the status quo is sovereign states. 

The following sections explore the current gestures, as limited as they are, to engage with states, and make the case for adding an additional prong to this engagement strategy - measures to restrict state sovereignty. 

Anatomy of an X-risk Mitigation Policy[2]
Phase 1 - Epistemics: Understand what problems exist, which ones are most important, and which ones it is best for the community to work on.
Phase 2 - Technical problem-solving: Devise solutions to the problems identified in Phase 1.
Phase 3 - Advocacy (Legislation): Push for solutions to be legislated nationally and globally. 
Phase 4 - Advocacy (Enforcement): Push for legislation to be implemented.  

States as Important Actors in Safeguarding Existential Security

Since the birth of the modern nation-state (the dates of which are debateable), the state has, until very recently, been the sole actor capable of producing and deploying weapons of mass destruction (nuclear weapons being a helpful example). Of course, with the proliferation of emerging technologies which democratise the ability to cause devastating harm, non-state actors have become increasingly capable of causing widespread destruction.


  1. Non-state actors still primarily operate within the borderly confines of sovereign states, and hence under their jurisdictions.
  2. States still exert considerable direct over the development and production of technologies which may catastrophic harm (e.g. nuclear weapons and bio-weapons).

    Most importanty:
  3. Remedies and mitigation strategies for such problems rely on states to implement them.

In the academic community, this third point has gain increasing salience. For many scholars, national sovereignty seems increasingly ineffective in tackling global problems. As one theorist of sovereignty puts it, ‘Vulnerability in the face of new risks is not something that modifies only legal sovereignty, but also operational sovereignty; in other words, the capacity of the states to assert themselves in ordinary political affairs’.  Others have argued that the project of global security is coming under the increasing strain of restrictions put on it by the imperative of state sovereignty. More anecdotally, conversations with several senior members of the EA community have yielded a similar conclusion - state sovereignty is an underconsidered obstacle to combatting global risks. 

For a demonstration of this principle, one need not look further back than the uneven efforts to combat the coronavirus. The notion of ‘epidemic sovereignty’ - that states have sovereign control over their efforts to combat epidemics as they see fit - has massively hamstrung the capacity of the WHO to carry out a concerted, global effort to combat Covid-19. For some thinkers, the multilateral system for the WHO that underscores largely voluntary cooperation ‘failed at what it had been specifically redesigned in 2005 to do: prevent the denial and inaction of one nation from putting many other nations at risk of a pandemic of deadly disease’. 

This ‘redesign’ in 2005 of the WHO’s relationship with states occurred in the immediate aftermath of the WHO’s failure in compelling China to deal with the SARS epidemic in a timely and appropriate fashion. However, its policy of ‘naming and shaming’ has been ineffective in the current crisis because there is no incentive for states to comply. In fact, states can actively depend on the WHO for continued assistance despite not following guidelines due to the salient goal of eliminating the virus everywhere so it does not re-emerge anywhere else. Naturally, the problem of state unwillingness is likely to be orders of magnitude more important in cases where, to continue with this example, a viral pathogen can wipe out billions of people.

The next section examines what actions the EA commnity has undertaken with respect to states. 

A Brief Look at the EA Community's Actions in Relation to States

There can be four reasons why states are unable or unwilling to work on mitigation of existential risks, which broadly falls in line with my typology of strategies undertaken by the community.

Reasons for States not to comply to x-risk reduction strategiesSuggested category of remedies pushed by the EA Community
Category 1 - Epistemic: 'We don't know what the problems are, and we definitely don't know how to solve them'. Here states have a knowledge problem, which precludes them from mitigating existential risks. Technical Problem-Solving[3]: The community expends considerable resources in defining and publicising existential risks, and researches remedies to existential risks (e.g. solving the alignment problem, dual-use research, universal vaccines etc). 
Category 2 -Capabilites: 'We know what the problems are and we have have a general idea of how to solve them, we just don't have enough technical and financial resources to put behind the problem.' Here many states, in particular poor states, suffer from a capabilities problem and require financial and technical help.  Capacity-building: The community expends resources which either seek to affect the problem directly or help capacitate states and international institutions with the funding and expertise needed to implement remedies to existential risks. 
Category 3 - Incentives:  'We know what the problems are and we get why you want to solve them, we just don't want to because its too difficult, it'll cost too much, it's not domestically popular etc. We need better reasons to do this'. Here states do not have enough incentives to commit to x-risk reducation strategies and require persuasion. Incentive-building: The community expends resources towards lobbying governments to take up actions which could mitigate risks, through campaign finance, electoral competition, and socializing remedies on international platforms. 
Category 4 - Non-compliance:  'We don't want to do it, full stop.'?

The EA community's efforts have evolved during the last four years from an almost singular focus on Category 1 towards greater recognition of the important of convicing and capacitating governments, lending more focus towards Categories 2 and 3. Category 4 has remained almost completely unexplored, with very few exceptions.[4] 

The following section outlines some historical and hypothetical case-studies which frame the urgency of the problem. They fall under broadly four categories: 

  1. States attempt to build weapons or technologies that could cause widespread global destruction.
  2. States harbour individuals who intend to build weapons or technologies that could cause widespread global destruction.
  3. States partake in actions that contradict strategies to mitigate existential risks.
  4. States partake in actions that increase the likelihood of existential risks (e.g. make dangerous pronouncements against other states, spread anti-science thinking, engage in offensive capacity-building, attack other states).

Some Examples of Concern

'Hands off our Samples!' - Indonesia, 2008

From July 2005 to December 2007, Indonesia reported the highest number of influenza A (H5N1) human cases in the world: 116 cases, with an extremely high fatality rate of 81 percent. The Indonesian government barred research scientists in the country from sharing these samples with international laboratories, potentially putting the entire region at risk of an epidemic. 

'We will eat grass or leaves' - Pakistan, 1998

After India announced its decision to build nuclear weapons, Pakistan's future Prime Minister Zulfiqar Ali Bhutto declared in 1965, 'If India builds the bomb, we will eat grass or leaves, even go hungry, but we will get one of our own.' Pakistan's decision has made the South Asian region among experts the uncontroversially most likely hotspot for a nuclear confrontation. 

The following statements are from world leaders whose countries were worst afflicted during the COVID-19 pandemic - one can easily imagine how species-threatening this behaviour would be it took place in the context of pandemic that is orders of magnitude worse in mortality and infection terms. Despite PHEIC guidlines issues from the WHO, states consistently and explicitly rejected these precepts, endangering not just their own constituents but those across the planet. 

‘It stopped COVID, it stopped everything.’ 

US President Donald Trump, inspecting a section of concrete wall on the US-Mexico border, 23 June 2020 (Covid Death Count: ~200,000). 

"I caught the virus and took hydroxychloroquine. Maybe I'm the only head of state who looked for that remedy... I'm not going to back down, I'm stubborn, I'm persistent." 

Brazilian President Jair Bolsonaro, upon criticism regarding promoting dangerous and unverified treatments, July 2021 (Covid Death Count: ~484,000). 

“The party unequivocally hails its leadership for introducing India to the world as a proud and victorious nation in the fight against Covid.” 

Statement by National Executive of India’s Ruling Party BJP, weeks before the global spread of the Delta Variant, February 2020 (Covid Death Count: ~250,000)

Some Hypothetical Cases in Brief: 

  1. State X decides to invest in dangerous biological research (or research on artifical intelligence) or researchers with bad incentives move to this territory to conduct this research and develop these kinds of weapons, knowing they will not be prosecuted.
  2. State Y refuses to bring into force rules, institutions and/or technologies that could contribute towards the mitigation of existential risks. This may be the various technical solutions proposed by the EA community.
  3. At the onset of a global crisis, State Z refuses to comply with suggested responses that could mitigate it and prevent it from becoming species-threatening.

An Aside of Interest: Why do people not write on this in the community? 

While I do not want to detract from the argumentative thrust of the post too much, I think it is important to briefly ponder over why state sovereignty has been rarely touched upon in community discussions, insofar as they could shed light on structural problems with discourse in EA which go beyond this particular critique. The following reasons come to mind after discussions on this questions with several EAs. The first two are answers I find somewhat suspect, followed by answers which I deem more likely. 

Unlikely Answers

  1. It is a bad priority to focus on. I find this to be one of the least compelling answers to the question. This is not because there aren't valid objections to this thesis (the next section responds to more than half a dozen and there are likely many more), but that seems to imply that this thesis is so fundamentally bad that its not even worth talking about, especially when the forum and EA discussion groups are rife with half-baked 'bad' ideas we are all eager to share, stress-test. and receive feedback on, on the off chance that the ideas are helpful, or at least engineer helpful discussion.
  2. It is already being worked on by non-EAs. This gets at the neglectedness question and I think this too is an unlikely answer. For one, as stated in 1, there are hundreds of posts on the Forum which cover issues not neglected by non-EAs. Secondly, and more importantly, very, very few people in the nuclear, biological and chemical space explictly work on mechanisms that could restrict state sovereignty. Where state sovereignty has been relevant has been almost solely in the context of mass human rights abuses and acts of war, leaving the space linking global catastrophic risks to state sovereignty almost completely unexplored. This sets up the EA community - as one that is willing to think about bold ideas or bold spins on old ideas courageously - as the ideal environment for this kind of discussion.

Likely Answers

  1. Didn't think about it/ don't work on political scientific or legal aspects of global policy. This answer comes somewhat close to what I think is the truest answer. I am sure that hundreds of EAs are globally informed regarding issues which involve state sovereignty on some level (Ukraine just being the most recent tragic example) but, as many EA on the forum have posited employing a range of heuritistics, it takes a lot more mental bandwidth and focus to work through an idea and its implications for your research, especially when it belongs to a discipline outside your expertise and is hence non-obvious to you. This is even more true for ideas which seem so ingrained in the global system so as to be taken as natural defaults.
  2. Do not have enough political scientists/activists/global policy makers in EA. As almost a continuation of 1, I think this is the most likely answer to be true. This point - and the larger point about the dearth of non-quant EAs - has been more eloquently expressed in multiple other parts of the Forum, but suffice it to say that our enormous focus on solving technical problems, paying people lots of money to get them to do things, and convincing people using rational arguments as the main problem-solving strategies may have something to do with the movement's demographics.

Possible Objections to Working on State Sovereignty as an EA Priority

There are a variety of objections to working on this kind of research as a cause area. I would like to once again thank numerous members of the community for their offer of objections to this thesis, which are summarized and responded to below. 

Before I respond to each cricisim specifically, I want to offer four generic responses to all possible objections: 

  1. This is the first step in thinking about this problem, and will by default likely have many more open questions associated with it than when it has been thought through further by many people. Hence, dismissing this problem outright may be ill-advised unless at least some bandwidth has been dedicated towards stress-testing the contours of the problem further. Sometimes, it may be imprudent to argue against starting work on a problem because it seems too difficult, if it is that difficult because we do not work on it.
  2. This post is intended to be very brief and should not be treated as the most robust thesis even I can put forward in defense of this work. I estimate that making a robust argument for restricting state sovereignty as integral to mitigating existential risks would at minimum require a book-length research project, and likely the follow-up of several other works.
  3. Thinking about this problem does not necessarily imply taking a stance, per se, on state sovereignty. At minimum, I am simply advocating that it is a crucial consideration that should be factored into the production and advocacy of policies designed to mitigate existential risks. Equally importantly, one can concede that state sovereignty is an inherent good (or at least, that its restriction is a bad) and still make this argument by framing it is a trade-off between competing priorities.[5]
  4. Even if this problem is near impossible to resolve, I believe more research dedicated towards state sovereignty delivers dividends to the community. Among many, these are the inclusion of more political scientific and legal experts in the community, much-needed greater discussion of the role of intergovernmental organizations in mitigating existential risks, and simply more discussion of the political dynamics which underpin global policy, the understanding of which is vital towards advocating for x-risk reduction strategies.

The specific responses to criqitues are summarized in the table below.[6] 

Criticism Response
This is completely unworkable and untractable as a cause area. 

Investigating this as a cause area is not equivalent to advocating publicly for restrictions on state sovereignty. 

Secondly, if research demonstrates that restrictions on state sovereignty are critical for mitigating x-risks in certain arenas, then tractability is not grouns for a definitive rejection  i.e. it is not an absoluate concern. 

This will lead to the great power problem, i.e. big countries will have no reason to respect restrictions on state sovereignty.There is some historical precedent for powerful states respecting international strictures. There is also precedent for them being dealt punishment if they don't. Apart from the more obvious examples in non-proliferation and rules of war, the WTO is an interesting example: the US has lost nearly half, and China a third, of all its cases, and dished out billions of dollars to small countries. But these rulings are respected due to a more general acceptance of the system of free trade. A similar understanding may be nurtured here. 
This will lead to the weak power problem i.e. restrictions will be used as a mechanism to selectively prosecute poor countries. Even conceding this completely, as horrible as a proposition as this might seem, it may still be worth the trade-off in stopping high-consequence events i.e. selective justice is not worse than no justice in the context of x-risk (we get some and not others). In addition, it is unclear if poor states are likely to be hotspots in any case, and if the 'WMD' rhetoric from Iraq curries as much domestic favour for interventions as in the past. 
Even if you are only advocating for restrictions  to do with x-risks, there is likely to be a mission creep e.g. there are several economic, social and political dimensions to many x-risk mitigation policies which means a restriction of sovereignty won't just apply to a specific policy, but likely be very wide-ranging. This is probably true for some policies e.g.dual-use and the importance of a bio-economy, but I don't know if this is true for many other important ones (e.g. bio-weapons, LAWs production). I would also posit that while emerging technlogies are definitely much more value-agnostic than the following example, we place restrictions on state activities which may boost economic growth all the time (e.g. the small arms trade treaty). 
This is essentially advocacting for a global government, which is incredibly unlikely. Not necessarily. We were able to establish a (broadly respected) set of strictures on human rights and rules of war after WWII that did not require a global government. In fact, both of those existing discourses can likely be marshaled to push for support in this arena as well due to their close relevance. 
Setting up an architecture which restricts state sovereignty may incentivize states to be non-transparent with their involvement in technological development proliferation. This is true for essentially all current restrictions on technology in status quo. In addition, we may only need a couple of countries to be 'caught red-handed' for the cost of deterrance to be high.
This is an unnecessarily aggressive suggestion. Cooperating with states is the best pathway towards impact.Recent history has abundant evidence that this is not the case. Even if this is completely true, we should, in the spirit of the community, stress-test all policy strategies for ourselves before rejecting them wholesale. 


The following section can be taken up be thought of as belonging in four buckets: 

  1. Real-world suggestions that could be used as straw models for debate.
  2. EA measures in terms of setting up funding and cause areas structures.
  3. Legal observations which are helpful case-studies for the history of restricting state sovereignty.
  4. Political developments which make it easier to make the case for restricting state sovereignty.

Real-world Suggestions

  1. An independent unit within the United Nations which can investigate concerning technlogical development and send investigation teams across the world.
  2. The United Nations Secretary General employing a special legal mechanism to investigate technological development and recommend sanctions to the United Nations Security Council.
  3. The ratification of a 'Universal Declaration against Extinction' akin to the Universal Declaration for Human Rights, which enshrines state responsibility against irresponsible technological development within their sovereign borders.

EA Measures

  1. Setting up a 'future of state sovereignty' working group which brings together an interdisciplinary group of researchers variously focused on international law, political science, political history, global activism, international economics, public policy and other relevant disciplines to investigate the linkage between state sovereignty and mitigation of existential risks.
  2. Setting up a fund for promising young researchers who work on state sovereignty as an obstacle to mitigation of existential risks.
  3. Organize conferences with members of the academic community and policy community who focus on this issue, alongside experts in the existential risk community.[8]
  4. Lobby for these discussions to feature in UNSG reports such as Our Common Agenda, and at global conferences such as the UN Summit for the Future.

Legal Observations 

  1. Exceptional Circumstances Trumping State Sovereignty: Several legal precedents of international law restrict the sovereignty of a nation-state in extraordinary circumstances. Some of these are well-known, including violations of the UN Charter and significant violations of the UDHR, CWC and NPT, but there others. For instance, the Bonn Agreement (2001) and the accompanying UNSC Resolution 1386 significantly limited the legal sovereignty of Afghanistan. The UNSC has also mandated taking action against Libya via Resolution 1973, deeming the Libyan state not to be sovereign due to the ‘responsibility to protect’ principle (see 'political developments' section below). The UN Secretary General on the Reform of the UNO (2017) outlined an ‘exalted notion of security’ within which ecological catastrophes and widespread disease were to be treated as matters of international security (also see 'conclusion' below).
  2. Individual Criminal Liability Trumping State Sovereingty: The most striking example of this is the set of rulings issued by the International Military Tribunal of 1946 in the aftermath of the Second World War, more widely known as the Nuremberg Trials. One of the authors of the judgement(s) proclaimed that ‘The entire law relating to war crimes, as distinct from the crime of war, is based upon the principle of individual responsibility’. The preamble of the judgements noted that ‘The very essence of the {Nuremberg} charter is that individuals have international duties which transcend the national obligations of obedience imposed by the individual state’. The ratification of the Universal Declaration of Human Rights has given legal permanence to this shift. Articles 29(1) and (2) place international obligations on individuals, largely in the context of respecting the human-rights of others. The recent prosecution of prominent leaders by the International Criminal Court seems to have furthered this principle of individual liability (see for instance, the cases against Sudan’s Omar AlBashir, Kenya’s Uhuru Kenyatta and Congo’s Thomas Lubunga). Prosecuting individuals for behaviour which increases existential risks could logically extend this principle of individual responsibility and override state sovereignty.

Political Observations

  1.  A general erosion of sovereignty: For some observers, it seems that civilizational structures have been constantly scaling up, going from villages to cities, cities to states, states to regionalism, and beyond. It only seems, it is posited, to be a matter of time before the national sovereign state is replaced with a global structure that represents the next step in humanity’s civilizational integration. An extrapolation of this trend seems to signal the creation of a ‘global singleton’ in the coming decades. Even if this claim is overblown, states have definitely become less sovereign since WWII. The rise of transnational corporations (TNCs) which trasncend borders and many times subvert or resist the authority of sovereign states, the rise of intergovernmental organizations such as the UN, the ascendance of supranational unions such as the EU, and just the basic reality of living in an economically globalized system with complex interconnectedness has made states much less sovereign they were before and it is a bargain, caveating for some populist backlash, that most states have accepted.
  2. Human rights discourse: While some have argued that ‘the subset of human rights that global governance institutions legally articulate or enforce must not undermine the political autonomy or sovereign equality of polities’, discourses surrounding human-rights and counter-terrorism - once at odds with each other - have come to a fascinating convergence in arguing that sovereignty is not a right but a privilege. Some have gone even further to define human-rights as ‘everything that justifies international interference’. Former Secretary-General Ban KiMoon, in a statement co-signed by previous occupants of his office declared that ‘the evolution of international human-rights standards and support for their implementation has now reached the stage where norms of non-intervention, and the related deference to sovereignty rights, no longer apply to the same extent in the face of severe human rights abuses’. Insofar as an x-risk or GCR event implies widespread violations of human rights, such discourse may be valuable for future consideration.
  3. Responsibility to Protect and Self-Defense: The legal discourse on sovereignty has been dominated by the ‘Responsibility to Protect’ principle: sovereignty is not an inherent right but a responsibility vested in governments to protect their citizens from mass atrocities, entailing that states can be considered non-sovereign if they violate this responsibility as a result of being unsuccessful in preventing - or indeed perpetuating - mass atrocities against their citizens. While there is much debate about the precise parameters of the R2P doctrine and indeed regarding its practicalities, the principle itself has been affirmed to some extent by all great powers and the majority of states. The weight of this principle exponentially increases if one factors in the possibility of an atrocity that isn’t just international, but global and transgenerational: in other words, existential. Even if one does not accept R2P and espouses the more restrictive ‘self-defense’ principle, an argument to intervene in the affairs of other states can be possibly defensible.

Conclusion - Our 'Most Important Century' for Restrictions on State Sovereingty?

In September, 2021 the United Nations Secretary General issued his special report, titled 'Our Common Agenda'[9], a by-product of fantastic lobbying and advocacy by many members of the x-risk community including Toby Ord. The report explicitly mentions 'existential risks', 'long-termism', and future generations. It sets up the grounds for the upcoming 'Summit of the Future', and advocates for many instruments, including: 

  • A Futures Lab for futures impact assessments and “regularly reporting on megatrends and catastrophic risks”
  • A Special Envoy for Future Generations to assist on “long-term thinking and foresight” and explore various international mechanisms for representing future generations, including...
  • Repurposing the Trusteeship Council to represent the interests of future generations (a major but long-inactive organ of the UN)
  • A Declaration on Future Generations
  • An Emergency Platform to convene key actors in response to complex global crises
  • A Strategic Foresight and Global Risk Report to be released every 5 years

These developments, occurring both because of and in tandem with increasing discussions on the failues of global governance amidst the COVID-19 pandemic, not only open up a window for more discussion of and funding towards EA priorities, but also an opportunity to push forward a discussion the role that states play as our main actors of (in)action on the global stage. 

This moment is not altogether dissimilar from the end of WW2, and the next decade of international legal and political thinking, advocacy and reform can set up an architecture that can mitigate and respond to our most pressing challenges in the future. I believe the EA community can play a crucial role in this fight. 

  1. ^

    I would like to thank Dr. Catherine Rhodes and David Manhiem for supervising this body of research. I would also like to thank numerous attendees at EAG SF, members of EA DC, colleagues at the Nuclear Threat Initiative and numerous others for their candid and extensive feedback on this idea. 

  2. ^

    Naturally, this is a straw flow-chart of the process, and may be missing some important elements. For instance, there a variety of reasons why this flow-chart is more applicable to certain risks (e.g. Bio) over others (e.g. AI). It also simplifies the first two phases which could come about through complex processes of policy socialization, policy transfer, policy competition and others. 

  3. ^

    Note that I find this to be roughly 90% of what the community's efforts are focused on. While 2 and 3 still get more traction and attention then 4, the subject of this paper, all of them are easily eclipsed by the focus on technical-problem solving. I hope to write a future post on this priority concentration generally, and feedback on this point is welcome. 

  4. ^

    An example of an exception would be the 'Joint Assessment Mechanism' adovcated for the Nuclear Threat Initiative, which would give the UN Secretary General the mandate to send investigation teams into countries to investigate the microbial origins of high-consequence biological events. 

  5. ^

    As the citizen of a country which has seen the worst of what the interventionism of the US military industrial complex has to offer, I am particularly sensitive to this criticism and count myself as one of the people in this camp.

  6. ^

    To keep this post brief, I have opted for citing broader over more narrow critiques e.g. which get at specific legal and policy difficulties. I am happy to share those separetely with anyone who is interested. 

  7. ^

    I should note that I am not married to any of the legal observations and political developments as particularly worthy courses of study or examples to guide futures courses of action - this is precisely why we need more work dedicated to this body of research. 

  8. ^

    It was partly in this stead that I organized the LSE Future of Humanity Summit, and am hosting the forthcoming Harvard Future of Humanity Summit.

  9. ^

    A fantastic summary of this report relevant to the EA community is available here, from which much of this section is borrowed. 





More posts like this

Sorted by Click to highlight new comments since:

Super interesting topic! Very neglected in EA

More from HTC
Curated and popular this week
Relevant opportunities