Thanks for highlighting these links! I've now read them and found them interesting. I have a few somewhat vague/broad questions I'd be interested to hear your reactions to:
Do you have thoughts on how impactful the Treaty on the Prohibition of Nuclear Weapons has been or will be?
Do you have thoughts on how impactful treaties intended to reduce existential risk from AI could be, and what the best form for them to take is?
These treaties could focus on things like reducing the chance of corner-cutting/race dynamics, incentivising/requiring better alignment/safety methods, reducing risks of misuse, etc.
Do you have thoughts on what sorts of questions it'd be useful to have forecasts about, in order to inform answers to those questions?
These could be proxies we'll get answers about in the next couple years, or further afield and more directly important things.
I ask because I may look into one or both of the first two of those questions later this year, and will probably be involved in setting up a batch of nuclear-risk-related forecasting questions for the site Metaculus soon.
But as I say, those questions are somewhat vague/broad, and I already know of other sources, contacts, etc. - so please don't feel compelled to respond unless you find those questions interesting or already have pre-loaded thoughts.
One piece of info related to the NPT that might be helpful: The NPT does contain an article (article VI) in which the the P5 (the 5 current permanent members of the UN Security Council, who at the time the NPT was made were the only countries who had successfully tested nuclear weapons) as well as all of the parties agree to participate in good-faith negotiations to pursue nuclear disarmament, but it does not specify a time-table and the language is deliberately vague. I think the NPT has done a good job of doing what its main goal is and what its name implies: limiting nuclear proliferation. It has clearly not been able to get existing nuclear powers to get rid of all of their nuclear weapons (although it's hard to know if the NPT has not played a significant role in reducing the arsenals of the nuclear powers relative to a counterfactual world in which there was no NPT... perhaps in a counterfactual world without the NPT, the US and USSR would not feel compelled to engage in the SALT and START negotiations without officially committing in Article VI of the NPT). Luisa has done a lot more thinking about the Nuclear Ban treaty, so I'll defer to her on that.
Jeffrey Ohl is going to look into this question as part of a summer project for the Stanford Existential Risk Initiative, and I'll be mentoring him. We literally just started talking about the project last week, so more to come on that! For now, I'll say the characteristics of the technology itself that you are trying to regulate (e.g. fissile material vs. inputs to the creating of AGI for instance) is very important in terms of how a successful treaty could be constructed. This aspect is important in terms of the mechanisms that must be put in place to verify compliance with the treaty. I co-wrote an article with Chris Bakerlee on engineered pathogens for Vox a few years ago that discusses, among other things, some of the challenges around regulating biotechnology that make verifying compliance with the Biological Weapons Convention difficult https://www.vox.com/future-perfect/2018/12/6/18127430/superbugs-biotech-pathogens-biorisk-pandemic
What time range are you looking at when it comes to forecasts, and what sort of things do you have in mind? I recall when I read Superforecasting a few years ago that forecasts aren't particularly reliable beyond a few years even for Superforecasters (though correct me if I'm wrong/maybe views on that are different now than they were then?). These treaties operate on pretty long time-scales... e.g. the NPT was conceived of in the mid 1960s, it was signed in 1968, it went into force in 1970, and then countries joined over the course of a few decades. https://en.wikipedia.org/wiki/List_of_parties_to_the_Treaty_on_the_Non-Proliferation_of_Nuclear_Weapons
I'll be quite interested to learn more about Jeffrey's project once it's further along. I might reach out to you or Jeffrey in a few weeks about that.
Regarding the forecasts, we can have any time range and any topic. I already have a bunch of ideas, but just wanted to see if anything bubbled to mind for you independently which I could add to my list. (It's ok if not!)
I recall when I read Superforecasting a few years ago that forecasts aren't particularly reliable beyond a few years even for Superforecasters (though correct me if I'm wrong/maybe views on that are different now than they were then?).
I guess this might depend what you mean by "particularly reliable".
My understanding is that there's basically just no good evidence either way regarding how accurate and calibrated forecasts are over long time-scales (at least if we restrict ourselves to relevant kinds of forecasts, e.g. ones made by people who seem to have been genuinely trying rather than just making claims for rhetorical/political effect). But there's a little evidence (from Tetlock) to suggest that accuracy may decline relatively slowly after the first year or so. See in particular the great post How Feasible Is Long-range Forecasting?, footnote 17 there, and posts tagged long-range forecasting. Here's the summary of that post:
How accurate do long-range (≥10yr) forecasts tend to be, and how much should we rely on them?
As an initial exploration of this question, I sought to study the track record of long-range forecasting exercises from the past. Unfortunately, my key finding so far is that it is difficult to learn much of value from those exercises, for the following reasons:
Long-range forecasts are often stated too imprecisely to be judged for accuracy. [More]
Even if a forecast is stated precisely, it might be difficult to find the information needed to check the forecast for accuracy. [More]
Degrees of confidence for long-range forecasts are rarely quantified. [More]
In most cases, no comparison to a “baseline method” or “null model” is possible, which makes it difficult to assess how easy or difficult the original forecasts were. [More]
Incentives for forecaster accuracy are usually unclear or weak. [More]
Very few studies have been designed so as to allow confident inference about which factors contributed to forecasting accuracy. [More]
It’s difficult to know how comparable past forecasting exercises are to the forecasting we [at Open Phil] do for grantmaking purposes, e.g. because the forecasts we make are of a different type, and because the forecasting training and methods we use are different. [More]
We plan to continue to make long-range quantified forecasts about our work so that, in the long run, we might learn something about the feasibility of long-range forecasting, at least for our own case. [More]
One other reason why I think that understanding the NPT is important for longtermists: As the world decarbonizes to address climate change (my other big area of research), nuclear electricity generation may increase substantially into more countries, and in particular to countries with lower levels of development/technology. It's crucial to know if the existing nonproliferation regime can ensure that this doesn't cause proliferation, and what sorts of investments must be made to ensure that nonproliferation regime continues to work.
Great post -- and I fully agree that understanding the success of the NPT is hugley interesting and promising, and I also agree with this comment on the importance of this question regarding nuclear energy deployment!
There seems to be one line of thinking according to which almost any facilitating of a country's starting a civilian nuclear energy programme increases proliferation risk and another line of thinking according to getting ahead of the curve in exporting comparatively proliferation-resistant technology might actually reduce proliferation risks in the longer term. The idea behind the second view is nicely expressed by Rebecca David Gibbons in this passage:
"The positive effects of nuclear assistance on nonproliferation suggest a second important policy lesson for the United States and its allies: attempt to regain and maintain a competitive nuclear industry. When US and allied technology is desirable, the nonproliferation regime benefits. Indonesia, Japan, Egypt, and several other states joined the NPT in part to receive nuclear technology from Western suppliers. Today, Egypt is purchasing its nuclear technology from Russia and China and has not agreed to the most stringent IAEA safeguards agreement, the Additional Protocol. If suppliers less concerned with nonproliferation have better technology or offer more favorable agreements than the United States and its allies, the nuclear nonproliferation regime could be weakened." (Gibbons 2020, p. 294)
From: Gibbons, R. D. (2020), Supply to deny: The benefits of nuclear assistance for nuclear nonproliferation, Journal of Global Security Studies, 5:282-298.
Could you outline whether you think this reasoning is compelling, Danny?
Yes, I think it is. There is a literature on whether nuclear assistance and technology sharing for peaceful uses tends to promote or hinder nuclear proliferation, that I mention and cite a bit in my second CSIS piece.
The NPT: Learning from a Longtermist Success [Links!]
Addressing global threats globally is critical to securing the long-run future. I argue that understanding the important and often overlooked Nuclear Nonproliferation Treaty (NPT) is crucial to understanding how future treaties and international regimes can be structured to counter global threats.
US Ambassador Llewellyn Thompson, left, signs the NPT as Soviet Foreign Minister Andrei Gromyko watches in Moscow, Russia, on July 1, 1968.
I recently wrote two articles for the think tank CSIS on the Nuclear Nonproliferation Treaty (NPT):
I am currently a PhD candidate at the Columbia University School of International and Public Affairs, a Global Priorities Fellow with the Forethought Foundation, and I did this work as part of the 2020 Nuclear Scholars program at CSIS. You can see more on my background here: https://rdanielbressler.com/.
Why I Think Understanding the Nuclear Nonproliferation Treaty is Important For Longtermists
In my view, one of the most crucial factors towards avoiding existential risk and achieving existential security is to achieve strong international cooperation on global issues. My very talented Uncle and I put together a cartoon that tries to convey this point, inspired by Ben Franklin’s famous cartoon:
Effective global cooperation is critical to securing the long-run future (Image created by C.M. Butzer and R. Daniel Bressler).
In practice, achieving cooperation is often elusive because the interests of individual countries on critical issues often differ from what is best for humanity as a whole (my advisor Scott Barrett has a very good book that describes this general class of issues: https://www.google.com/books/edition/Why_Cooperate/zNYTDAAAQBAJ?hl=en&gbpv=0). To understand how international cooperation can be achieved in the future, it is important to understand if, how, when, and under what conditions it has been successful in the past.
In my view, the Nuclear Nonproliferation Treaty (the NPT for short) is the best historical example of successful multilateral international cooperation on an issue that is hard. Nearly every country in the world is a party to the treaty, making it among the most widely subscribed treaties in the world. In its 50-year history, only one country (North Korea) has withdrawn even though the treaty has an article specifying that countries can leave. The treaty requires countries to conclude comprehensive safeguards agreements that open them to International Atomic Energy Agency (IAEA) inspection. No country has successfully been able to produce nuclear weapons while subject to the NPT and its safeguards.
My articles provide an overview of the current scholarship that examines if and how the NPT has limited nuclear proliferation (read them to see the details!). I believe that the NPT has limited nuclear proliferation, perhaps significantly, but I think there remains important work to understand how and why the NPT continues to work and under what conditions it may no longer work. I am currently working on a project with Scott Barrett to address this question by constructing a game-theoretical model of the treaty.
There are a wide array of potential global catastrophic risks such as threats related to biosecurity and climate change that are many and increasing in importance. Understanding if, how, and why the NPT has reduced the risk of nuclear proliferation is crucial to understanding how future treaties and international regimes may be structured to counter these other global threats.
A Side Note
If you are interested in issues related to nuclear weapons, nuclear nonproliferation, or nuclear energy, I would highly recommend doing the CSISProject on Nuclear IssuesNuclear Scholars program that I did this past year. They are generally looking for people that have some background experience/knowledge in nuclear issues (you can see the list of scholars in my cohort here), so I would suggest gaining some of this experience/knowledge ahead of time to be a competitive applicant.
This is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan?
Summary
Rising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions.
Public Opinion...
I think right now EAs might be making a significant mistake by paying insufficient attention to the political realm. As EAs we tend to figure out what’s most impactful for us to work on and focus hard. That’s great! But there are various actions that are ‘non-delegatable’ - the extent to which an individual can do the action is limited (like voting, going to a protest, making hard money contributions to particular campaigns). It might be useful if we were all more in the habit of doing variou...
New Video from AI in Context: The Fall and Rise of Sam Altman
If you want to skip straight to the video, here it is!
AI in Context is excited to be back with our fourth video! For those just hearing from us, we make videos for 80,000 Hours, telling stories about transformative AI...
Thanks for highlighting these links! I've now read them and found them interesting. I have a few somewhat vague/broad questions I'd be interested to hear your reactions to:
I ask because I may look into one or both of the first two of those questions later this year, and will probably be involved in setting up a batch of nuclear-risk-related forecasting questions for the site Metaculus soon.
But as I say, those questions are somewhat vague/broad, and I already know of other sources, contacts, etc. - so please don't feel compelled to respond unless you find those questions interesting or already have pre-loaded thoughts.
Thanks for this response!
I'll be quite interested to learn more about Jeffrey's project once it's further along. I might reach out to you or Jeffrey in a few weeks about that.
Regarding the forecasts, we can have any time range and any topic. I already have a bunch of ideas, but just wanted to see if anything bubbled to mind for you independently which I could add to my list. (It's ok if not!)
I guess this might depend what you mean by "particularly reliable".
My understanding is that there's basically just no good evidence either way regarding how accurate and calibrated forecasts are over long time-scales (at least if we restrict ourselves to relevant kinds of forecasts, e.g. ones made by people who seem to have been genuinely trying rather than just making claims for rhetorical/political effect). But there's a little evidence (from Tetlock) to suggest that accuracy may decline relatively slowly after the first year or so. See in particular the great post How Feasible Is Long-range Forecasting?, footnote 17 there, and posts tagged long-range forecasting. Here's the summary of that post: