EA is associated with maximising expected utility. Which leads to counterintuitive recommendations --- such as (repeatedly) pressing a button that destroys the world with 99.99% chance but has a 1% chance of creating a (10 000+)-times better world. I claim that:
- the assumptions underlying expected utility maximisation do not apply to these scenarios,
- it's ok if you don't want to press that button,
- it's ok if EA is not about pressing that button,
- [and aaaaaah, for f...s sake, stop associating EA with pressing that button].[1]
More detailed explanation:
An argument for maximising expected utility goes something like this:
Suppose you were risk averse --- for example, suppose you prefferred (A) a >90% chance of $1 to (B) a 1% chance of $1000. Well, then I expose you to a series of 100 bets like this, and [apply probability theory] suddenly we are comparing (A') a >90% chance of $90 with (B') a >90% chance of $100K. Presumably you don't feel so happy about choosing (A) now, eh? Maybe you want to go back on this risk aversion thing, and start maximising expected utility with every decision, like a proper Utilitarian?
However, this argument critically relies on repeating the bets. If there is only ever one bet, or a small number of them, Central Limit Theorem does not apply!
In particular:
- In your personal life, there is nothing incoherent about being risk averse with respect to your overall happiness, taken over your whole life.
- In terms of near-term social impact, there is nothing incoherent about being risk averse with respect to the planetary scale.
- In longtermism, there is nothing incoherent about being risk averse with respect to the scale of the future trajectories of the universe.
[Also: uuuhm, maaaybe don't rely on things that you don't understand, particularly if they start telling you to jump off a cliff?]
Disclaimer: There is a hypothetical version of this post that includes references, is polished, isn't a rant, has a better example of risk aversion, makes all the disclaimers such as "yeah yeah, I know most people's opinions are more nuanced than what I make them", and that I ran through Granterly a few times. Sadly, that version is less fun, more work, and doesn't exist. So apologies to all offended readers.
Edit: changed the title to better reflect the main claim.
- ^
Insert countless links here. Including by philosophers much smarter than me, who really should know better.
Yes, the expected utility is larger. The claim is that there is nothing incoherent about not maximising expected utility in this case.
To try rephrasing: Principle 1: if you have to choose a X% chance of getting some outcome A, and a >=X% chance of a strictly better outcome B, you should take B. Principle 2: if you will be facing a long series of comparably significant choices, you should decide each of them based on expected utility maximisation. Principle 3: you should do expected utility maximisation for every single choice. Even if that is the last/most important choice you will ever make.
The claim is that: P1 is solid. P2 follows from P1 (via Central Limit Theorem, or whatever math). But P3 does not follow from P1/P2, and there will be cases where it might be justified to not obey P3. (Like the case with 51% chance of doubling the world's goodness, 49% chance of destroying it.)
Note that I am not claiming it's wrong to do expected utility maximisation in all scenarios. Just saying that it both doing and not doing it is OK. And therefore it is (very?) non-strategic to associate your philosophical movement with it. (Given that most people's intuitions seem to be against it.)
Does this explanation make sense? Maybe I should change the title to something with expected utility?