Last year Holden Karnofsky wrote the post, “EA is about maximization, and maximization is perilous”. You could read the post, but I suggest you just jump on board because Holden is cool, and morality is hard.

Given that you now believe that maximisation of doing good is actually bad and scary, you should also probably make some adjustments to the classic thought experiment you use to get your friends on board with the new mission of “do the most good possible [a large but not too large amount of good] using evidence and reason”.

A slightly modified drowning child thought experiment goes as follows

Imagine that you are walking by a small pond, and you see five children drowning. You can easily save the child without putting yourself in great danger, but doing so will ruin your expensive shoes. Should you save the children?

Obviously, your first instinct is to save all the children. But remember, maximisation is perilous.^[1] It’s this kind of attitude that leads to atrocities like large financial crimes.^[2] Instead, you should just save three or four of the children. That is still a large amount of good, and importantly, it is not maximally large.

But what should you do if you encounter just one drowning child? The options at first pass seem bleak – you can either:

• Ignore the child and let them drown (which many people believe is bad).
• Save the child (but know that you have tried to maximise good in that situation).

I think there are a few neat solutions to get around these moral conundrums:

Save the child with some reasonable probability (say 80%).

Before wading into the shallow pond, whip out the D10 you were carrying in your backpack. If you roll an eight or lower, then go ahead and save the child. Otherwise, go about your day.

Only partially save the child

You may have an opportunity to help the child to various degrees. Rather than picking up the child and then ensuring that they find their parents or doing other previously thought as reasonable things, you could:

• Move the child to shallower waters so they are only drowning a little bit.
• Help the child out of the water but then abandon them somewhere within a 300m radius of the pond.
• Create a manifold market on whether the child will be saved and bid against it to incentivise other people to help the child.

The QALY approach^[3]

• Save the child but replace them with an adult who is not able to swim (but is likely to have fewer years of healthy life left).
• Commit now to a policy of only saving children who are sufficiently old or likely to have only moderately healthy/happy lives.

The King Solomon approach

• Cut the child in half and save the left half of them from drowning

Using these approaches, you should be able to convey the optimal most Holden-approved amount of good.

1. ^{^}

    If you like, you can remember the heuristic “maximisation bad”.

2. ^{^}

    As well as other things like eradicating diseases.

3. ^{^}

    QALYs are quality-adjusted life years (essentially a metric for healthy years lived).