Agreed that counting paths at a full multiple would seem silly.
My proposed fix is a bit different. It cares about immediate welfare plus the welfare of future types of observer-moments, where futures are weighted by how likely we are to continue into each one, so each time step adds up to a weight of one. In your example, splitting shares the future between the copies and merging adds the shares back up, so the merged mind’s future isn’t multiplied by 2^1000.
Each brief copy does count as its own observer-moment with its own future, so the splitting adds a little. But that grows with the number of experiences that actually happened, not exponentially.
I don’t go into the fix in this post. Post was mainly to raise the question.
I agree with you that B is much worse. But I disagree that standard utilitarianism, as it is formulated now, reaches our conclusion that B is worse than A.
In the thought experiment, merging into one mind doesn’t add any negative experiences. The hell mind’s experiences are exactly the same in both options. It's still just one mind suffering in hell, and that seems to be all standard utilitarianism counts.
Rather than just counting immediate experience, I think it makes more sense to also account for subjective continuity, i.e. that multiple minds lead into the one mind in hell.
I think that analogy doesn’t quite fit. The road doesn’t have a point of view, but people do. Two people headed to hell seems worse than one.
My question is really “if merging were possible, would we merge everybody into hell?” Copying minds didn’t seem possible a decade ago, but assuming digital sentience is a thing, it now seems quite routine. If we wouldn’t merge everybody into hell, then there are some unresolved issues in utilitarianism.
I recently proposed a view I call Markov utilitarianism. To me, it feels like a natural way to think about ethics. I came to it after thinking about how I care about my future, and how I really care about how I feel right now, from the first-person perspective.
I care about my actual subjective futures, weighted by how likely I am to find myself in each, rather than the physical futures someone looking from a third-person perspective implicitly treats as relevant to me. (Physical futures and my subjective futures might come apart because of copying and merging.)
I care about how I feel right now because of the contents of my experience, rather than any elusive "number of subjectively identical copies" someone looking from a third-person perspective would associate with me.
This is prudence, in the philosophical sense of rational concern for one's own wellbeing. And it has exactly the same form as the equation reinforcement learning uses for an agent's value of being in a state, the Bellman equation (hence 'Markov'):
V(t) = r(t) + γ Σ P(t → t') V(t') over types of experience t.
To make it an ethics that considers everybody, we universalize it and sum impartially across all unique perspectives, mine and everyone else's:
U = Σ V(t)
In slogan form: "Prudence is caring about how you feel right now, and about each possible next experience in proportion to how likely you are to find yourself there. Ethics is doing that for every qualitatively unique perspective, counting each once".
Or more precisely: "Count each qualitatively unique perspective once, and care about it as it would care about how it feels now and what it will experience next, if fully informed about its futures".
Markov utilitarianism is closely related to total utilitarianism, but it structurally prefers lives that get better over time, and once minds can be copied and merged, it avoids conclusions like it being permissible to merge people into a being fated for extreme suffering.
So it feels like a natural extension of how you already care about yourself, extended to every perspective. Starting an ethics from this first-person point of view seems better to me because it avoids the third-person assumptions hidden in total utilitarianism.
Posting this subsection from the main post because it got asked about in the comments:
"What about near-identical copies?"
Exact copies count once but the smallest difference in feeling between them makes them near-copies, which would count twice. This is a jump, which some might consider unacceptable.
It doesn't seem to me that any view that counts observer-moments fully can avoid a jump. For instance, Bostrom's insulator can be gradually inserted, which means that at some point the token view has to say that one experience became two, and that this is morally relevant, which is over a physical change nobody feels. Bostrom himself ends up allowing fractional numbers of experiences, which doesn't seem to reflect how experience works. Where the physics separates is a judgment call, whereas where the contents of experience differ isn't, which seems like the better place for a jump.
A view could avoid a jump by discounting smoothly with similarity. For instance, the Saturation View of MacAskill and Tarsney (2026) does something like this, discounting near-duplicate lives toward a bound. However, this needs a similarity metric that we've decided is important. Any similarity metric seems to be something we choose from the outside. And I think we should recognize that "near identical" assumes a lot - and might be appealing to the sense that they can't be that different. Yet, you can win the lottery by a difference of a single digit.
Discounting for similarity would also fail the test that we should "restate every value claim in terms of what some being feels or will feel" (Bar, 2026). This is because neither copy feels how similar it is to the other. The same argument doesn't apply to exact copies, since counting one's feelings would count everything about the other's feelings. So, replication neutrality agrees with the Saturation View that filling the world with exact replicas of one existence would not, by itself, be the best use of resources. However, it gets to this position without valuing variety (which is something we judge from the outside) and unlike the Saturation View, it counts near-duplicates fully (on the basis of their experience).
More objections and responses are in the full post.
This is the near-identical copies objection. I think the post makes the case better than I can in comments, so I'll point you to this section, under the heading "What about near-identical copies?".
Thanks for asking around - agreed it needs justification. A fuller case is in this section of the full post.
I suspect the answers depend a lot on framing. Described from the outside, it's 100 shock events. From the inside, what you go through is exactly the same either way. If ethics is about experience, I think we ought to prefer the first-person framing over the third-person one.
The trade makes this concrete, i.e. a half-strength shock run 100 times with full resets, versus a full-strength shock run once. From the inside, the only difference is how much it hurts, never how many times it's run in exactly the same way.
From the inside, A1 and A100 are the same. My first-person experience in both is:
I go about my day.
I get put in the chair.
I get shocked.
I get out of the chair.
I go about my day again.
I never experience more than one shock, so I'd consider A1 and A100 to have the same disvalue.
All else equal, if the shock could be half as painful, and if I have to be shocked either way, then I'd happily take any number of identical runs of it, as long as they really are identical - my memory is fully reset, I don't age, the shocks and everything else I experience are exactly the same. It would just mean that step 3 (the shock) is less painful for me.
If instead each run continued subjectively from the last, for example if I remembered each previous shock, so that I experienced 2 -> 3 -> 4 -> 2 -> 3 -> 4 -> ... -> 2 -> 3 -> 4 in sequence, it would be a different case. Those runs wouldn't even be identical anymore since each would have memories of the last. Then I'd agree A100 is much worse.
Thanks - I think we actually agree on the chicken case. My view doesn't weigh by similarity. Instead, it's about identical subjective experiences. In other words, I'm counting at the level of types of experiences - individuated by the unique content experience has - what it's like from the inside.
If there is any qualitative difference at all, experiences count separately. So, near copies of any experience count fully.
Two "A"s written on a blackboard are one type with two tokens. Writing a "B" adds a second type, no matter how close "A" and "B" might seem to us, and each type counts in full.
This is also why the view isn't Eigenism, which weighs concern by similarity to you. I argue in the full post in this subsection that any similarity metric is imposed from the outside. It's not fundamental to experience. If I'm reading Bentham's Bulldog's post correctly, it targets that similarity weighting, which my view does not have.
Agreed that counting paths at a full multiple would seem silly.
My proposed fix is a bit different. It cares about immediate welfare plus the welfare of future types of observer-moments, where futures are weighted by how likely we are to continue into each one, so each time step adds up to a weight of one. In your example, splitting shares the future between the copies and merging adds the shares back up, so the merged mind’s future isn’t multiplied by 2^1000.
Each brief copy does count as its own observer-moment with its own future, so the splitting adds a little. But that grows with the number of experiences that actually happened, not exponentially.
I don’t go into the fix in this post. Post was mainly to raise the question.
I agree with you that B is much worse. But I disagree that standard utilitarianism, as it is formulated now, reaches our conclusion that B is worse than A.
In the thought experiment, merging into one mind doesn’t add any negative experiences. The hell mind’s experiences are exactly the same in both options. It's still just one mind suffering in hell, and that seems to be all standard utilitarianism counts.
Rather than just counting immediate experience, I think it makes more sense to also account for subjective continuity, i.e. that multiple minds lead into the one mind in hell.
I think that analogy doesn’t quite fit. The road doesn’t have a point of view, but people do. Two people headed to hell seems worse than one.
My question is really “if merging were possible, would we merge everybody into hell?” Copying minds didn’t seem possible a decade ago, but assuming digital sentience is a thing, it now seems quite routine. If we wouldn’t merge everybody into hell, then there are some unresolved issues in utilitarianism.
Markov utilitarianism as universalized prudence
I recently proposed a view I call Markov utilitarianism. To me, it feels like a natural way to think about ethics. I came to it after thinking about how I care about my future, and how I really care about how I feel right now, from the first-person perspective.
This is prudence, in the philosophical sense of rational concern for one's own wellbeing. And it has exactly the same form as the equation reinforcement learning uses for an agent's value of being in a state, the Bellman equation (hence 'Markov'):
V(t) = r(t) + γ Σ P(t → t') V(t') over types of experience t.
To make it an ethics that considers everybody, we universalize it and sum impartially across all unique perspectives, mine and everyone else's:
U = Σ V(t)
In slogan form: "Prudence is caring about how you feel right now, and about each possible next experience in proportion to how likely you are to find yourself there. Ethics is doing that for every qualitatively unique perspective, counting each once".
Or more precisely: "Count each qualitatively unique perspective once, and care about it as it would care about how it feels now and what it will experience next, if fully informed about its futures".
Markov utilitarianism is closely related to total utilitarianism, but it structurally prefers lives that get better over time, and once minds can be copied and merged, it avoids conclusions like it being permissible to merge people into a being fated for extreme suffering.
So it feels like a natural extension of how you already care about yourself, extended to every perspective. Starting an ethics from this first-person point of view seems better to me because it avoids the third-person assumptions hidden in total utilitarianism.
Posting this subsection from the main post because it got asked about in the comments:
More objections and responses are in the full post.
This is the near-identical copies objection. I think the post makes the case better than I can in comments, so I'll point you to this section, under the heading "What about near-identical copies?".
Thanks for asking around - agreed it needs justification. A fuller case is in this section of the full post.
I suspect the answers depend a lot on framing. Described from the outside, it's 100 shock events. From the inside, what you go through is exactly the same either way. If ethics is about experience, I think we ought to prefer the first-person framing over the third-person one.
The trade makes this concrete, i.e. a half-strength shock run 100 times with full resets, versus a full-strength shock run once. From the inside, the only difference is how much it hurts, never how many times it's run in exactly the same way.
I'm picturing this.
From the inside, A1 and A100 are the same. My first-person experience in both is:
I never experience more than one shock, so I'd consider A1 and A100 to have the same disvalue.
All else equal, if the shock could be half as painful, and if I have to be shocked either way, then I'd happily take any number of identical runs of it, as long as they really are identical - my memory is fully reset, I don't age, the shocks and everything else I experience are exactly the same. It would just mean that step 3 (the shock) is less painful for me.
If instead each run continued subjectively from the last, for example if I remembered each previous shock, so that I experienced 2 -> 3 -> 4 -> 2 -> 3 -> 4 -> ... -> 2 -> 3 -> 4 in sequence, it would be a different case. Those runs wouldn't even be identical anymore since each would have memories of the last. Then I'd agree A100 is much worse.
Thanks - I think we actually agree on the chicken case. My view doesn't weigh by similarity. Instead, it's about identical subjective experiences. In other words, I'm counting at the level of types of experiences - individuated by the unique content experience has - what it's like from the inside.
If there is any qualitative difference at all, experiences count separately. So, near copies of any experience count fully.
Two "A"s written on a blackboard are one type with two tokens. Writing a "B" adds a second type, no matter how close "A" and "B" might seem to us, and each type counts in full.
This is also why the view isn't Eigenism, which weighs concern by similarity to you. I argue in the full post in this subsection that any similarity metric is imposed from the outside. It's not fundamental to experience. If I'm reading Bentham's Bulldog's post correctly, it targets that similarity weighting, which my view does not have.
Decided to call it "Markov utilitarianism"
Not because it sounds cooler (it does) but because it is a utilitarianism and it does contrast with the other views by being an MRP.