Summary

• Historical terrorist attack deaths suggest the probability of a terrorist attack causing human extinction is astronomically low, 4.35*10^-15 per year according to my preferred estimate.
• One may well update to a much higher extinction risk after accounting for inside view factors. However, extraordinary evidence would be required to move up sufficiently many orders of magnitude for an AI or bio terrorist attack to have a decent chance of causing human extinction.
• I think it would be great if Open Philanthropy published the results of their efforts to quantify biorisk if/once they are available.
• In the realm of the more anthropogenic AI, bio and nuclear risk, I personally think underweighting the outside view is a major reason leading to overly high risk. I encourage readers to check David’s series exaggerating the risks, which includes subseries on climate, AI and bio risk.

Introduction

I listened to Kevin Esvelt’s stories on The 80,000 Hours Podcast about how bioterrorism may kill billions. They sounded plausible, but at the same time I thought I should account for the outside view, which I knew did not favour terrorism being a global catastrophic risk. So I decided to look into historical terrorist attack deaths to estimate a prior probability for a terrorist attack causing human extinction.

Methods

Firstly, I retrieved data for the deaths in the 202 k terrorist attacks between January 1970 and June 2021 on the Global Terrorism Database (GTB). Then I computed some historical stats for such deaths.

Secondly, I relied on the Python library fitter to find the distributions which best fit to the top 1 %:

• Terrorist attack deaths.
• Logarithm of the terrorist attack deaths as a fraction of the global population^[1].

I fitted the distributions to the 1 % most deadly attacks because I am interested in the right tail, which may decay faster than suggested by the less extreme points. fitter tries all the types of distributions in SciPy, 111 on 13 November 2023. For each type of distribution, the best fit is that with the lowest residual sum of squares (RSS), respecting the sum of the squared differences between the actual and predicted probability density function (PDF). I set the number of bins to define the PDF to the square root of the number of data points, and left the maximum time to find the best fit parameters to the default value in fitter of 30 s.

I estimated the probability of the terrorist attack deaths as a fraction of the global population being at least 10^-10, 10^-9, …, and 100 % multiplying:

• 1 %, which is the probability of the attack being in the right tail.
• Probability of the deaths as a fraction of the global population being at least 10^-10, 10^-9, …, and 100 % if the attack is in the right tail, which I got using the best fit parameters outputted by fitter.

I obtained the annual probability of terrorist attack extinction from 1 - (1 - “probability of the terrorist attack deaths as a fraction of the global population being at least 100 %”)^“terrorist attacks per year”.

I aggregated probabilities from different best fit distributions using the median. I did not use:

• The mean because it ignores information from extremely low predictions, and overweights outliers.
• The geometric mean of odds nor the geometric mean because many probabilities were 0.

The calculations are in this Sheet and this Colab.

Results

The results are in the Sheet.

Historical terrorist attacks stats

Terrorist attacks basic stats

Terrorist attacks by severity

Tail distribution of the terrorist attack deaths

Terrorist attacks tail risk

Below are the median RSS, coefficient of determination^[2] (R^2), probability of terrorist attack deaths as a fraction of the global population being at least 10^-10, 10^-9, …, and 100 %, and annual probability of a terrorist attack causing human extinction. The medians are taken across the best, top 10, and top 100 distributions according to 3 fitness criteria, lowest RSS^[3], Akaike information criterion (AIC), and Bayesian information criterion (BIC). My preferred estimate is the median taken across the 10 distributions with lowest RSS, which is the default fitness criterion in fitter, fitted to the top 1 % logarithm of the terrorist attack deaths as a fraction of the global population. The results for AIC and BIC are identical^[4]. Null values may be exactly 0 if they concern bounded distributions, or just sufficiently small to be rounded to 0 due to finite precision. I also show the annual probability of terrorist attack extinction as a function of R^2.

Best fit to the top 1 % terrorist attack deaths

Best fit to the top 1 % logarithm of the terrorist attack deaths as a fraction of the global population

Discussion

Historical terrorist attack deaths suggest the probability of a terrorist attack causing human extinction is astronomically low. According to my preferred estimate, the median taken across the 10 distributions with lowest RSS fitted to the top 1 % logarithm of the terrorist attack deaths as a fraction of the global population, the annual probability of human extinction caused by a terrorist attack is 4.35*10^-15. Across all my 18 median estimates, it is 4.35*10^-15 for 2, and 0 for the other 16. For context, in the Existential Risk Persuasion Tournament (XPT), superforecasters and domain experts^[5] predicted a probability of an engineered pathogen causing human extinction by 2100 of 0.01 % and 1 %, which are much higher than the implied by my prior estimates.

I do not think anthropics are confounding the results. There would be no one to do this analysis if a terrorist attack had caused human extinction in the past, but there would be for less severe large scale attacks, and there have not been any. The most deadly attack since 1970 on GTB only killed 1.70 k people. As a side note, the most deadly bioterrorist incident only killed 200 people (over 5 years).

Interestingly, some best fit distributions led to an annual probability of human extinction caused by a terrorist attack higher than 10 % (see last 2 graphs), which is unreasonably high. In terms of the best fit distributions to:

• The top 1 % terrorist attack deaths, a lognormal results in 45.1 %^[6] (R^2 = 14.5 %).
• The logarithm of the top 1 % terrorist attack deaths as a fraction of the global population (the following are ordered by descending R^2):
  • A half-Cauchy results in 39.5 % (R^2 of 94.8 %).
  • A skew-Cauchy results in 39.4 % (94.8 %).
  • A log-Laplace results in 28.8 % (90.8 %).
  • A Lévi results in 96.2 % (77.9 %).
  • A noncentral F-distribution results in 58.0 % (77.8 %).
  • A Cauchy results in 15.1 % (70.7 %).
  • A von Mises results in 100 % (66.4 %).
  • A F-distribution results in 66.4 % (65.5 %).

Nonetheless, I have no reason to put lots of weight into the distributions above, so I assume it makes sense to rely on the median. In addition, according to extreme value theory, the right tail should follow a generalised Pareto, whose respective best fit to:

• The top 1 % terrorist attack deaths results in an annual probability of human extinction caused by a terrorist attack of 0 (R^2 of 94.5 %).
• The logarithm of the top 1 % terrorist attack deaths as a fraction of the global population results in 0 (98.1 %).

Of course, no one is arguing that terrorist attacks with the current capabilities pose a meaningful extinction risk. The claim is that the world may become vulnerable if advances in AI and bio make super destructive technology much more widely accessible. One may well update to a much higher extinction risk after accounting for inside view factors, and indirect effects of terrorist attacks (like precipitating wars). However, extraordinary evidence would be required to move up sufficiently many orders of magnitude for an AI or bio terrorist attack to have a decent chance of causing human extinction. On the other hand, extinction may be caused via other routes, like a big war or accident.

I appreciate it is often difficult to present compelling evidence for high terrorism risk due to infohazards, even if these hinder good epistemics. However, there are ways of quantifying terrorism risk which mitigate them. From Open Philanthropy’s request for proposal to quantify biorisk:

One way of estimating biological risk that we do not recommend is ‘threat assessment’—investigating various ways that one could cause a biological catastrophe. This approach may be valuable in certain situations, but the information hazards involved make it inherently risky. In our view, the harms outweigh the benefits in most cases.

A second, less risky approach is to abstract away most biological details and instead consider general ‘base rates’. The aim is to estimate the likelihood of a biological attack or accident using historical data and base rates of analogous scenarios, and of risk factors such as warfare or terrorism.

As far as know, there is currently no publicly available detailed model of tail biorisk. I think it would be great if Open Philanthropy published the results of their efforts to quantify biorisk if/once they are available. The request was published about 19 months ago, and the respective grants were announced 17 months ago. Denise Melchin asked 12 months ago whether the results would be published, but there has been no reply.

In general, I agree with David Thorstad that Toby Ord’s guesses for the existential risk between 2021 and 2120 given in The Precipice are very high. In the realm of the more anthropogenic AI, bio and nuclear risk, I personally think underweighting the outside view is a major reason leading to overly high risk. I encourage readers to check David’s series exaggerating the risks, which includes subseries on climate, AI and bio risk.

Acknowledgements

Thanks to Anonymous Person 1 and Anonymous Person 2 for feedback on the draft.

1. ^{^}

    fitter has trouble finding good distributions if I do not use the logarithm, presumably because the optimisation is harder / fails with smaller numbers.

2. ^{^}

    Fraction of the variance of the actual PDF explained by the predicted PDF. I computed this fraction from 1 minus the ratio between the residual and total sum of squares.

3. ^{^}

    Equivalent to lowest R^2.

4. ^{^}

    Although the results I present are identical, the rankings for AIC and BIC are not exactly the same.

5. ^{^}

    The sample drew heavily from the Effective Altruism (EA) community: about 42% of experts and 9% of superforecasters reported that they had attended an EA meetup.

6. ^{^}

    In addition, the best fit normal-inverse Gaussian to the top 1 % terrorist attack deaths results in 100 % (R^2 = 86.0 %). I did not mention this in the main text because there must be an error in fitter (or, less likely, in SciPy's norminvgauss) for this case. The respective cumulative distribution function (CDF) increases until terrorist attack deaths as a fraction of the global population of 10^-7, but then unexpectedly drops to 0, which cannot be right.