- In order to tell you something you didn't already know, a model must sometimes be surprising
- Object level: An ethical system that doesn't ever differ from your intuition is not more useful than just a gut feeling and gut feelings are unreliable
- There are very compelling responses to the repugnant conclusion, but I think the most compelling is to just accept it as both intuitively unappealing and true.
If you're unfamiliar with the repugnant conclusion, see the Stanford Encyclopedia of Philosophy article on the subject.
Useful models are surprising
Useful models are surprising. If a model is never surprising, then it hasn't told you anything new. (More formally: if it doesn't cause you to update on either answer or confidence, then you haven't learned anything). Evolution, Pangea, and climate change are all somewhat surprising. Because they're both true and surprising, learning them is useful. Utilitarianism is a useful tool for evaluating moral situations because it sometimes surprises you. If it never disagreed with your gut, then it wouldn't be helpful.
The repugnant conclusion is surprising, but I think that's actually a good sign: it means that utilitarianism can sometimes surprise you, which means (if it's true), that it's probably useful.
The corollary is more important: beware ethical systems without repugnant conclusions because they're not telling you anything new. They totally agree with your gut, and your gut is unreliable.
Reality is sometimes surprising
True models are ones that accurately represent reality. Reality is often surprising, so we should expect true models to surprise us often.
If useful models are surprising, and useful models are true, then it means that being surprising does not mean that a model is false. (Of course, in the absence of other evidence, you should be more skeptical of surprising models).
This isn't the same as saying surprising models are useful. I only intend this to rebut the claim "the repugnant conclusion is a counter-example to act utilitarianism because it gives the incorrect answer." In practice, I think this claim often is actually saying "the repugnant conclusion is a counter-example to act utilitarianism because it gives an answer that doesn't seem right intuitively."
Historically, anyway, and to small children. If these don't surprise you, I suspect it's because you've already accepted them as true (many years or decades ago)