Research project idea: Impact assessment of nuclear-risk-related orgs, programmes, movements, etc.

This post is part of a series of rough posts on nuclear risk research ideas. I strongly recommend that, before you read this post, you read the series’ summary & introduction post for context, caveats, and to see the list of other ideas. One caveat that’s especially worth flagging here is that I drafted this in late 2021 and haven’t updated it much since. I’m grateful to Will Aldred for help with this series.

One reason I'm publishing this now is to serve as one menu of research project ideas for upcoming summer research fellowships.

Some tentative bottom-line views about this project idea

What is this idea? How could it be tackled?

Many of the organizations, programmes, movements, policies, “intermediate goals” (see Aird & Aldred, 2022), interventions, etc. that might be worth supporting to reduce nuclear risk already have a track record and/or are similar to things that have track records. So our decisions about what to support could be informed by the empirical evidence those track records provide. In particular, for each thing we’re considering supporting, we could consider questions like:

• To what extent have that thing or things in its reference class achieved their intended objectives in the past?
• E.g., to what extent did ICAN increase the likelihood or pace of the ratification of the TPNW?
• What other positive or negative effects has the thing or things in its reference class had?
• E.g., did ICAN spread misleading or oversimplified ideas about nuclear risk? Did national security policymakers perceive its work negatively in a way that will make them less supportive of other civil society nuclear risk reduction efforts? If so, to what extent?
• E.g., did participating in ICAN-related activities help people gain career capital for other useful work?
• How much of various resources (especially money and person-years) did the thing or things in its reference class consume in the past?
• E.g., what has ICAN’s budget and number of staff been each year of its operations?
• What do the positive and negative effects of the thing’s intended objectives appear to have been?
• E.g., to what extent does the ratification of the TPNW appear to have increased or decreased nuclear risk, improved or worsened prospects for global governance or international cooperativeness more broadly, etc.?^[1]

These questions could be tackled via activities such as:

• Investigating case studies using the sorts of methods common among historians or investigative journalists
• Gathering quantitative data, making plausible Fermi estimates, and/or constructing quantitative models of the benefits, harms, and costs of various efforts
• Reading existing impact assessments or similar things (e.g., progress updates, history books), including from the relevant organisations etc. themselves
• Emailing, interviewing, or surveying people who were involved in relevant organisations etc., who were involved in funding them, or who have for some other reason formed views on the effects of their activities
• Other activities common in impact assessments
• Unfortunately I don’t know a huge amount about impact assessment myself

This is really more like a type of project than a specific project idea; which specific organisations, programmes, movements, etc., one focuses on would substantially affect what impacts this project would have, how long it’d take, how best to pursue it, etc.

Note that some insights from this project could also be useful for decisions about whether and how to try to try to influence various countries’ foreign, national security, or technology policies in contexts other than nuclear risk (e.g., for AI risk or biosecurity).

What sort of person might be a good fit for this?

I expect any good generalist researcher could provide a useful analysis of these questions. I expect someone to be a stronger fit the more experience they have with history research, investigative journalism, cost-effectiveness modelling, or impact assessment, or the more they’re the sort of person who’d be interested in and good at using those methodologies.

Some relevant previous work

• Sempere’s (2021) “Shallow evaluations of longtermist organizations”
• MacArthur’s (2020) “Nuclear Challenges Big Bet: 2020 Evaluation Report”
• Rodriguez’s (2019) “Will the Treaty on the Prohibition of Nuclear Weapons affect nuclear deproliferation through legal channels?
• Bressler’s (2021) “The NPT: Learning from a Longtermist Success” and links in that piece
• Rubinson’s (2021) “Philanthropy, Nuclear Nonproliferation, and Threat Reduction”
• Longley’s (2020) “Carnegie Corporation of New York Among Nuclear Security Funders Who ‘Punch Above Their Weight’”

I could also probably share on request some unpolished notes and further links on the TPNW in particular.

1. ^{^}

    See also Rodriguez (2019).