Suppose someone tries to convince you to give them $5, with the following argument:

Pascal’s Mugger: “I’m actually a traveller from another dimension, where 10^10^10^10 lives are in danger! $^{1}$ Quick, I need $5 to save those lives, no time to explain! I don’t expect you to believe me, of course, but if you follow the expected utility calculation, you’ll give me the money—because surely the probability that I’m telling the truth, while tiny, is at least 1-in-10^10^10^10.” $^{2}$

Now suppose someone tries to convince you to donate $5 to their organization, with the following argument:

Long-termist: $^{3}$ “My organization works to prevent human extinction. Admittedly, the probability that your donation will make the difference between success and failure is tiny, but if it does, then you’ll save $10^{40}$ lives! $^{4}$ And surely the probability is greater than 1-in- $10^{40}$ . So you are saving at least 1 life, in expectation, for $5—much better than the Against Malaria Foundation!”

Obviously, we should refuse the Mugger. Should we also refuse the long-termist, by parallel reasoning? In general, do the reasons we have to refuse the Mugger also count as reasons to de-prioritize projects that seem high-expected-value but have small probabilities of success—projects like existential risk reduction, activism for major policy changes, and blue-sky scientific research?

This article explores that question, along with the more general issue of how tiny probabilities of vast utilities (i.e. vast benefits) should be weighed in our decision-making. It draws on academic philosophy as well as research from the effective altruism community. $^{5}$

Summary

There are many views about how to handle tiny probabilities of vast utilities, but they all are controversial. Some of these views undermine arguments for mainstream long-termist projects and some do not. However, long-termist projects shelter within the herd of ordinary behaviors: It is difficult to find a view that undermines arguments for mainstream long-termist projects without also undermining arguments for behaviors like fastening your seatbelt, voting, or building safer nuclear reactors.

Amidst this controversy, it would be naive to say things like “Even if the probability of preventing extinction is one in a quadrillion, we should still prioritize x-risk reduction over everything else…”

Yet it would also be naive to say things like “Long-termists are vic...

Fanaticism