Effective Altruism Forum
EA Forum

Comment Permalink

In practice, I think the effects of one's actions decay to practically 0 after 100 years or so. In principle, I am open one's actions having effects which are arbitrarily large, but not infinite, and continuity does not rule out arbitrarily large effects.

If you allow arbitrarily large values and prospects with infinitely many different possible outcomes, then you can construct St Petersburg-like prospects, which have infinite expected value but only take finite value in every outcome. These violate Continuity (if it's meant to apply to all prospects, including ones with infinitely many possible outcomes). So from arbitrary large values, we violate Continuity.

We've also discussed this a bit before, and I don't expect to change your mind now, but I think actually infinite effects are quite plausible (mostly through acausal influence in a possibly spatially infinite universe), and I think it's unwarranted to assign them probability 0.

Reality forces us to compare outcomes, at least implicitly.

There are decision rules that are consistent with violations of Completeness. I'm guessing you want to treat incomparable prospects/lotteries as equivalent or that whenever you pick one prospect over another, the one you pick is at least as good as the latter, but this would force other constraints on how you compare prospects/lotteries that these decision rules for incomplete preferences don't.

I just do not see how adding the same possibility to each of 2 lotteries can change my assessment of these.

You could read more about the relevant accounts of risk aversion and difference-making risk aversion, e.g. discussed here and here. Their motivations would explain why and how Independence is violated. To be clear, I'm not personally sold on them.

See in context

Maximising expected utility follows from self-evident premises?

by Vasco Grilo🔸

Jan 192 min read 25

17

PhilosophyClassical utilitarianismConsequentialismDecision theoryExpected valueRisk aversionUtilitarianismEthics of existential risk

Frontpage

This is a linkpost for https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem

This is a linkpost for Von Neumann–Morgenstern utility theorem, which shows that one accepts 4 premises if and only if one maximises expected utility. In my mind, all the 4 premises are self-evident. So I do not see how one can reject maximising expected utility in principle. Relatedly, I think the Repugnant Conclusion follows from 3 self-evident premises.

In 1947, John von Neumann and Oskar Morgenstern proved that any individual whose preferences satisfied four axioms [premises] has a utility function, where such an individual's preferences can be represented on an interval scale [which "allows for defining the degree of difference between measurements"] and the individual will always prefer actions that maximize expected utility.^[1] That is, they proved that an agent is (VNM-)rational [has preferences satisfying the 4 axioms] if and only if there exists a real-valued function u defined by possible outcomes such that every preference of the agent is characterized by maximizing the expected value of u, which can then be defined as the agent's VNM-utility (it is unique up to affine transformations i.e. adding a constant and multiplying by a positive scalar). No claim is made that the agent has a "conscious desire" to maximize u, only that u exists.
[...]
Completeness assumes that an individual has well defined preferences:
Axiom 1 (Completeness) For any lotteries and $M$ , either $L ⪰ M$ or $M ⪰ L$ .
(the individual must express some preference or indifference^[4]). Note that this implies reflexivity.
Transitivity assumes that preferences are consistent across any three options:
Axiom 2 (Transitivity) If $L ⪰ M$ and $M ⪰ N$ , then $L ⪰ N$ .
Continuity assumes that there is a "tipping point" between being better than and worse than a given middle option:
Axiom 3 (Continuity): If $L ⪯ M ⪯ N$ , then there exists a probability $p \in [0, 1]$ such that $p L + (1 - p) N \sim M$
where the notation on the left side refers to a situation in which $L$ is received with probability $p$ and $N$ is received with probability $(1 - p)$ .
[...]
Independence assumes that a preference holds independently of the probability of another outcome.
Axiom 4 (Independence): For any $M$ and $p \in [0, 1)$ (with the "irrelevant" part of the lottery underlined):
$L ⪯ N$ if and only if $(1 - p) L + p M - -- - ⪯ (1 - p) N + p M - -- -$
In other words, the probabilities involving $M$ cancel out and don't affect our decision, because the probability of $M$ is the same in both lotteries.

17 Reactions

Comments25

Sorted by

New & upvoted

Click to highlight new comments since: Today at 9:47 PM

Anthony DiGiovanniJan 1914

Why do you consider completeness self-evident? (Or continuity, although I'm more sympathetic to that one.)

Also, it's important not to conflate "given these axioms, your preferences can be represented as maximizing expected utility w.r.t. some utility function" with "given these axioms [and a precise probability distribution representing your beliefs], you ought to make decisions by maximizing expected value, where 'value' is given by the axiology you actually endorse." I'd recommend this paper on the topic (especially Sec. 4), and Sec. 2.2 here.

Vasco Grilo🔸Jan 192

Hi Anthony,

I think completeness is self-evident because "the individual must express some preference or indifference". Reality forces them to do so. For example, if they donate to organisation A over B, at least implicitly, they imply donating to A is as good or better than donating to B. If they decide to keep the money for personal consumption, at least implicitly, they imply that is as good or better than donating.

I believe continuity is self-evident because rejecting it implies seemingly non-sensical decisions. For example, if one prefers 100 $ over 10 $, and this over 1 $, continuity says there is a probability p such that one is indifferent between 10 $ and a lottery involving a probability p of winning 1 $, and 1 - p of winning 100 $. One would prefer the lottery with p = 0 over 10 $, because then one would be certain to win 100 $. One would prefer 10 $ over the lottery with p = 1, because then one would be certain to win 1 $. If there was not a tipping point between preferring the lottery or 10 $, one would have to be insensitive to an increased probability of an outcome better than 10 $ (100 $), and a decreased probability of an outcome worse than 10 $ (1 $), which I see as non-sensical.

Anthony DiGiovanniJan 1912

Thanks! I'll just respond re: completeness for now.

When we ask "why should we maximize EV," we're interested in the reasons for our choices. Recognizing that I'm forced by reality to either donate or not-donate doesn't help me answer whether it's rational to strictly prefer donating, strictly prefer not-donating, be precisely indifferent, or none of the above.
Incomplete preferences have at least one qualitatively different property from complete ones, described here, and reality doesn't force you to violate this property.
Not that you're claiming this directly, but just to flag, because in my experience people often conflate these things: Even if in some sense your all-things-considered preferences need to be complete, this doesn't mean your preferences w.r.t. your first-order axiology need to be complete. For example, take the donation case. You might be very sympathetic to a total utilitarian axiology, but when deciding whether to donate, your evaluation of the total utilitarian betterness-under-uncertainty of one option vs. another doesn't need to be complete. You might, say, just rule out options that are stochastically dominated w.r.t. total utility, and then decide among the remaining options based on non-consequentialist considerations. (More on this idea here.)

Vasco Grilo🔸Jan 212

Thanks, Anthony.

2. Incomplete preferences have at least one qualitatively different property from complete ones, described here, and reality doesn't force you to violate this property.

I read the section you linked, and I understand preferential gaps are the property of incomplete preferences which you are referring to. I do not think preferential gaps make sense in principle. If one was exactly indifferent between 2 outcomes, I believe any improvement/worsening of one of them must make one prefer one of the outcomes over the other. At the same time, if one is roughly indifferent between 2 outcomes, a sufficiently small improvement/worsening of one of them will still lead to one being practically indifferent between them. For example, although I think i) 1 $ plus a chance of 10^-100 of 1 $ is clearly better than ii) 1 $, I am practically indifferent between i) and ii), because the value of 10^-100 $ is negligible.

3. Not that you're claiming this directly, but just to flag, because in my experience people often conflate these things: Even if in some sense your all-things-considered preferences need to be complete, this doesn't mean your preferences w.r.t. your first-order axiology need to be complete.

Both are complete for me, as I fully endorse expectational total hedonistic utilitarianism (ETHU) in principle. In practice, I think it is useful to rely on heuristics from other moral theories to make better decisions under ETHU. I believe the categorical imperative is a great one, for example, although it is very central to deontology.

Anthony DiGiovanniJan 222

To be clear, "preferential gap" in the linked article just means incomplete preferences. The property in question is insensitivity to mild sweetening.

If one was exactly indifferent between 2 outcomes, I believe any improvement/worsening of one of them must make one prefer one of the outcomes over the other

But that's exactly the point — incompleteness is not equivalent to indifference, because when you have an incomplete preference between 2 outcomes it's not the case that a mild improvement/worsening makes you have a strict preference. I don't understand what you think doesn't "make sense in principle" about insensitivity to mild sweetening.

I fully endorse expectational total hedonistic utilitarianism (ETHU) in principle

As in you're 100% certain, and wouldn't put weight on other considerations even as a tiebreaker? That seems extreme. (If, say, you became convinced all your options were incomparable from an ETHU perspective because of cluelessness, you would presumably still all-things-considered-prefer not to do something that injures yourself for no reason.)

Vasco Grilo🔸Jan 222

As in you're 100% certain, and wouldn't put weight on other considerations even as a tiebreaker?

Yes.

(If, say, you became convinced all your options were incomparable from an ETHU perspective because of cluelessness, you would presumably still all-things-considered-prefer not to do something that injures yourself for no reason.)

Injuring myself can very easily be assessed under ETHU. It directly affects my mental states, and those of others via decreasing my productivity.

MichaelStJulesJan 1913

I don't think any of the axioms are self-evident. FWIW, I don't really think anything is self-evident, maybe other than direct logical deductions and applications of definitions.

I have some sympathy for rejecting each of them, except maybe transitivity, which I'm pretty strongly inclined not to give up. (EDIT: On the other hand, I'm quite willing to give up the Independence of Irrelevant Alternatives, which is similar to transitivity.) I give weight to views that violate the axioms, under normative uncertainty.

Some ways you might reject them:

Continuity: Continuity rules out infinities and prospects with finite value but infinite expected value, like St Petersburg lotteries. If Continuity is meant to apply to all logically coherent prospects (including prospects with infinitely many possible outcomes), then this implies your utility function must be bounded. This rules out expectational total utilitarianism as a general view.
Continuity: You might think some harms are infinitely worse than others, e.g. when suffering reaches the threshold of unbearability. It could also be that this threshold is imprecise/vague/fuzzy, and we would also reject Completeness to accommodate that.
Completeness: Some types of values/goods/bads may be incomparable. Or, you might think interpersonal welfare comparisons, e.g. across very different kinds of minds, are not always possible. Tradeoffs between incomparable values would often be indeterminate. Or, you might think they are comparable in principle, but only vaguely so, leaving gaps of incomparability when the tradeoffs seem too close.
Independence: Different accounts of risk aversion or difference-making risk aversion (not just decreasing marginal utility, which is consistent with Independence).

Karthik TadepalliJan 204

Continuity doesn't imply your utility function is bounded, just that it never takes on the value "infinity", ie for any value it takes on, there are higher and lower values that can be averaged to reach that value.

MichaelStJulesJan 205

If your utility function can take arbitrarily large but finite values, then you can design a prospect/lottery with infinitely many possible outcomes and infinite expected value, like the St Petersburg paradox. Then you can treat such a prospect/lottery as if it has infinite actual value, and demonstrate violations of Continuity the same way you would with an outcome with infinite value. This is assuming Continuity applies to arbitrary prospects/lotteries, including with infinitely many possible outcomes, not just finitely many possible outcomes per prospect/lottery.

(Infinitary versions of Independence and the Sure-Thing Principle also rule out "unbounded" utility functions. See Russell & Isaacs, 2020.)

Karthik TadepalliJan 204

Yes, continuity doesn't rule out St Petersburg paradoxes. But i don't see how unbounded utility leads to a contradiction. can you demonstrate it?

MichaelStJulesJan 20*13

Assume your utility function is unbounded from above. Pick outcomes $x_{1}, x_{2}, . . .$ such that $u (x_{n}) \geq 2^{n}$ . Let your lottery $X$ be $x_{n}$ with probability $1 / 2^{n}$ . Note that $\sum_{n = 1}^{\infty} 1 / 2^{n} = 1$ , so the probabilities sum to 1.

Then this lottery has infinite expected utility:

E [u (X)] = \infty \sum n = 1 \frac{1}{2^{n}} u (x_{n}) \geq \infty \sum n = 1 \frac{1}{2^{n}} 2^{n} = \infty \sum n = 1 1 = \infty .

Now, consider any two other lotteries $A$ and $B$ with finite expected utility, such that $A ≺ B ≺ X$ . There's no way to mix $A$ and $X$ probabilistically to be equivalent to $B$ , because

E [u (p A + (1 - p) X)] = p E [u (A)] + (1 - p) E [u (X)] = p E [u (A)] + \infty = \infty > E [u (B)],

whenever $p < 1$ . For $p = 1$ , $E [p A + (1 - p) X] = E [u (A)] < E [u (B)]$ .

So Continuity is violated.

Karthik TadepalliJan 214

Got it, yes I agree now.

Vasco Grilo🔸Jan 214

Thanks, Michael! Nitpick, E((X)) in the 3rd line from the bottom should be E(u(X)).

MichaelStJulesJan 214

Thanks, fixed!

Vasco Grilo🔸Jan 202

Thanks, Michael.

1. Continuity: Continuity rules out infinities and prospects with finite value but infinite expected value, like St Petersburg lotteries. If continuity is meant to apply to all logically coherent prospects (as usually assumed), then this implies your utility function must be bounded. This rules out expectational total utilitarianism as a general view.
2. Continuity: You might think some harms are infinitely worse than others, e.g. when suffering reaches the threshold of unbearability. It could also be that this threshold is imprecise/vague/fuzzy, and we would also reject completeness to accommodate that.

In practice, I think the effects of one's actions decay to practically 0 after 100 years or so. In principle, I am open to one's actions having effects which are arbitrarily large, but not infinite, and continuity does not rule out arbitrarily large effects.

3. Completeness: Some types of values/goods/bads may be incomparable. Or, you might think interpersonal welfare comparisons, e.g. across very different kinds of minds, are not always possible. Tradeoffs between incomparable values would often be indeterminate. Or, you might think they are comparable in principle, but only vaguely so, leaving gaps of incomparability when the tradeoffs seem too close.

Reality forces us to compare outcomes, at least implicitly.

4. Independence: Different accounts of risk aversion or difference-making risk aversion (not just decreasing marginal utility, which is consistent with Independence).

I just do not see how adding the same possibility to each of 2 lotteries can change my assessment of these.

MichaelStJulesJan 204

In practice, I think the effects of one's actions decay to practically 0 after 100 years or so. In principle, I am open one's actions having effects which are arbitrarily large, but not infinite, and continuity does not rule out arbitrarily large effects.

Reality forces us to compare outcomes, at least implicitly.

I just do not see how adding the same possibility to each of 2 lotteries can change my assessment of these.

Vasco Grilo🔸Jan 212

Thanks, Michael.

If you allow arbitrarily large values and prospects with infinitely many different possible outcomes, then you can construct St Petersburg-like prospects, which have infinite expected value but only take finite value in every outcome. These violate Continuity (if it's meant to apply to all prospects, including ones with infinitely many possible outcomes). So from arbitrary large values, we violate Continuity.

Sorry for the lack of clarity. In principle, I am open to lotteries with arbitrarily large expected utility, but not infinite, and continuity does not rule out arbitratily large expected utilities. I am open to lotteries with arbitrarily many outcomes (in principle), but not to lotteries with infinitely many outcomes (not even in principle).

We've also discussed this a bit before, and I don't expect to change your mind now, but I think actually infinite effects are quite plausible (mostly through acausal influence in a possibly spatially infinite universe), and I think it's unwarranted to assign them probability 0.

I think empirical evidence can take us from a very large universe to an arbitrarily large universe (for arbitrarily strong evidence), but never to an infinite universe. An arbitrarily large universe would still be infinitely smaller than an infinite universe, so I would say the former provides no empirical evidence for the latter. So I am confused about why discussions about infinite ethics often mention there is empirical evidence pointing to the existence of infinity^[1]. Assigning a probability of 0 to something for which there is not empirical evidence at all makes sense to me.

There are decision rules that are consistent with violations of Completeness. I'm guessing you want to treat incomparable prospects/lotteries as equivalent or that whenever you pick one prospect over another, the one you pick is at least as good as the latter, but this would force other constraints on how you compare prospects/lotteries that these decision rules for incomplete preferences don't.

I have not looked into the post you linked, but you guessed correctly. Which constraints would be forced as a result? I do not think preferential gaps make sense in principle.

You could read more about the relevant accounts of risk aversion and difference-making risk aversion, e.g. discussed here and here. Their motivations would explain why and how Independence is violated. To be clear, I'm not personally sold on them.

Thanks for the links. Plato's section The Challenge from Risk Aversion argues for risk aversion based on observed risk aversion with respect to resources like cups of tea and money. I guess the same applies to Rethink Priorities' section. I am very much on board with risk aversion with respect to resources, but I still think it makes all sense to be risk neutral relative to total hedonistic welfare.

^{^}
From Bostrom (2011), "Recent cosmological evidence suggests that the world is probably infinite".

Karthik TadepalliJan 1912

Maximizing expected utility is not the same as maximizing expected value. The latter assumes risk neutrality, but vNM is totally consistent with maximizing expected utility under arbitrary levels of risk aversion, meaning that it doesn't provide support for your view expressed elsewhere that risk aversion is inconsistent with vNM.

The key point is that there is a subtle difference between maximizing a linear combination of outcomes, vs maximizing a linear combination of some transformation of outcomes. That transformation can be arbitrarily concave, such that we would end up making a risk averse decision.

Vasco Grilo🔸Jan 192

Thanks for the comment, Karthik! I strongly upvoted it. I have changed "expected value" to "expected utility" in this post, and updated to the following the last paragraph of the comment of mine you linked to.

I reject risk aversion with respect to impartial welfare (although it makes all sense to be risk averse with respect to money), as I do not see why the value of additional welfare would decrease with welfare.

David Mathers🔸Jan 2010

Is there any pattern of behaviour that couldn't be interpreted as maximizing utility for some utility function? If not, even if vNM is self-evident it's not actually much of a constraint.

Vasco Grilo🔸Jan 212

Great point, David! I strongly upvoted it. There are lots of possible utility functions, so I think VNM-rationality imposes very few constraints.

David Mathers🔸Jan 2110

Thanks.

A partly underlying issue here is that it's not clear that the consequentialist/non-consequentialist division is actually all that deep or meaningful if you really think about it. The facts about "utility" in a consequentialist theory, are plausibly ultimately just a kind of short-hand for facts about preferability between outcomes that could be stated without any mention of numbers/utility/maximizing (at least if we allow infinitely long statements). But for non-consequentialist theories, you can also derive a preferability relation on outcomes (where what you do is part of the outcome, not just the results of your action), based on what the theory says you should do in a forced choice. For at least some such theories that look "deontic", in the sense of having rights that you shouldn't violate, even if it leads to higher net well-being, the resulting preferability ranking might happen to obey the 4 axioms and be VNM-rational. For such a deontic theory you could then express the theory as maximizing a relevant notion of utility if you really wanted to (at least if you can cardinalize the resulting ordering of actions by prefertability, via looking at preferences between chance-y prospects I don't know enough to know if meeting the axioms guarantees you can do this.) So any consequentialist theory is sort of really a number/utility-free theory about preferability in disguise, and at least some very deontic feeling theories are in some sense equivalent to consequentialist theories phrased in terms of utility.

Or so it seems to me anyway, I'm certainly not a real expert on this stuff.

Vasco Grilo🔸Jan 212

Thanks, David! That makes sense to me.

NunoSempereJan 207

I am extremely sympathetic to vNM, but think it's not constructive. I think the world is too high-dimensional, and in some sense we are low compute agents in a high compute world. See here for a bit more background.

For example, there are lotteries L and M which are complex enough that a) I would express a strong preference if given enough time to parse it, b) the best option is not to actually choose between them but do something else.
For continuity, you can't necessarily know which p it is.
If you want to extract someone's utility function, this is an ~nlogn operation (using mergesort where each ordering step ellicits a numerical comparison). This line of research is interesting to me, but because of the expense it only works with enough buy in, which one may not have.

In practice, I think vNM works as an idealization of the values of a high or infinite compute agent, but because making it constructive is very expensive, sometimes the best action is not to go through with that but to fall back on heuristics or shortcuts, heuristics which you won't be sure of either (again, as low compute agents in a higher complexity world).

Vasco Grilo🔸Jan 204

Thanks, Nuño. I strongly endorse maximising expected welfare, but I very much agree with using heuristics. At the same time, I would like to see more cost-effectiveness analyses.