MichaelStJules's Quick takes

Future debate week topics?

1. Global health & wellbeing (including animal welfare) vs global catastrophic risks, based on Open Phil's classifications.
2. Neartermism vs longtermism.
3. Extinction risks vs risks of astronomical suffering (s-risks).
4. Saving 1 horse-sized duck vs saving 100 duck-sized horses.

I like the idea of going through cause prioritization together on the EA Forum.

I feel increasingly unsympathetic to hedonism (and maybe experientalism generally?). Yes, emotions matter, and the strength of emotions could be taken to mean how much something matters, but if you separate a cow and her calf and they’re distressed by this, the appropriate response for their sake is not to drug or fool them until they feel better, it’s to reunite them. What they want is each other, not to feel better. Sometimes I think about something bad in the world that makes me sad; I don't think you do me any favour by just taking away my sadness; I don't want to stop feeling sad, what I want is for the bad in the world to be addressed.

Rather than affect being what matters in itself, maybe affect is a signal for what matters and its intensity tells us how much it matters. Hedonism as normally understood would therefore be like Goodhart’s law: it ignores the objects of our emotions. This distinction can also be made between different versions of preference utilitarianism/consequentialism, as "satisfaction versions" and "object versions". See Krister Bykvist's PhD thesis and Wlodek Rabinowicz  and Jan Österberg, "Value Based on Preferences: On Two Interpretations of Preference Utilitarianism" (unfortunately both unavailable online to me, at least).

Of course, often we do just want to feel better, and that matters, too. If someone wants to not suffer, then of course they should not suffer. 

Related: wireheading, the experience machine, complexity of value.

The procreation asymmetry can be formulated this way (due to Jeff McMahan):

while the fact that a person's life would be worse than no life at all ... constitutes a strong moral reason for not bringing him into existence, the fact that a person's life would be worth living provides no (or only a relatively weak) moral reason for bringing him into existence.

I think it's a consequence of a specific way of interpreting the claim "an outcome can only be worse than another if it's worse for someone", where the work goes into defining "worse for A". Using "better" instead of "worse" would give you a different asymmetry.

This is a summary of the argument for the procreation asymmetry here and in the comments, especially this comment, which also looks further at the case of bringing someone into existence with a good life. I think this is an actualist argument, similar to Krister Bykvist's argument in 2.1 (which cites Dan Brock from this book) and Derek Parfit's argument on p.150 of Reasons and Persons, and Johann Frick's argument (although his is not actualist, and he explicitly rejects actualism). The starting claim is that your ethical reasons are in some sense conditional on the existence of individuals, and the asymmetry between existence and nonexistence can lead to the procreation asymmetry.

1. From an outcome in which an individual doesn't/won't exist, they don't have any interests that would give you a reason to believe that another outcome is better on their account (they have no account!). So, ignoring other reasons, this outcome is not dominated by any other, and the welfare of an individual we could bring into existence is not in itself a reason to bring them into existence. This is reflected by the absence of arrows starting from the Nonexistence block in the image above.

2. An existing individual (or an individual who will exist) has interests. In an outcome in which they have a bad life, an outcome in which they didn't exist would have been better for them from the point of view of the outcome in which they do exist with a bad life, so an outcome with a bad life is dominated by one in which they don't exist, ignoring other reasons. Choosing an outcome which is dominated this way is worse than choosing an outcome that dominates it. So, that an individual would have negative welfare is a reason to prevent them from coming into existence. This is reflected by the arrow from Negative existence to Nonexistence in the image above.

3. If the individual would have had a good life, we could say that this would be better than their nonexistence and dominates it (ignoring other reasons), but this only applies from outcomes in which they exist and have a good life. If they never existed, because of 1, it would not be dominated from that outcome (ignoring other reasons).

Together, 1 and 2 are the procreation asymmetry (reversing the order of the two claims from McMahan's formulation).

I never found psychological hedonism (or motivational hedonism) very plausible, but I think it's worth pointing out that the standard version — according to which everyone is ultimately motivated only by their own pleasure and pain — is a form of psychological egoism and seems incompatible with sincerely being a hedonistic utilitarian or caring about others and their interests for their own sake.

From https://www.britannica.com/topic/psychological-hedonism :

Psychological hedonism, in philosophical psychology, the view that all human action is ultimately motivated by desires for pleasure and the avoidance of pain. It has been espoused by a variety of distinguished thinkers, including Epicurus, Jeremy Bentham, and John Stuart Mill, and important discussions of it can also be found in works by Plato, Aristotle, Joseph Butler, G.E. Moore, and Henry Sidgwick.

Because its defenders generally assume that agents are motivated only by the prospect of their own pleasures and pains, psychological hedonism is a form of psychological egoism.

More concretely, a psychological hedonist who cares about others, but only based on how it makes them feel, would prefer to never find out that they've caused harm or are doing less good than they could, if it wouldn't make them (eventually) feel better overall. They don't actually want to do good, they just want to feel like they're doing good. Ignorance is bliss.

They could be more inclined to get in or stay in an experience machine, knowing they'd feel better even if it meant never actually helping anyone else.

That being said, they might feel bad about it if they know they're in or would be in an experience machine. So, they might refuse the experience machine by following their immediate feelings and ignoring the fact that they'd feel better overall in the long run. This kind of person seems practically indistinguishable from someone who sincerely cares about others through and based on their feelings.

Utility functions (preferential or ethical, e.g. social welfare functions) can have lexicality, so that a difference in category $A$ can be larger than the maximum difference in category $B$, but we can still make probabilistic tradeoffs between them. This can be done, for example, by having separate utility functions, $f_A:X\to R$ and $f_B:X\to R$ for $A$ and $B$, respectively, such that

• $f_A\left(x\right)-f_A\left(y\right)\ge 1$ for all $x$ satisfying the condition $P\left(x\right)$ and all $y$ satisfying $Q\left(y\right)$ (e.g. $Q\left(y\right)$ can be the negation of $P\left(y\right)$, although this would normally lead to discontinuity).
• $f_B$ is bounded to have range in the interval $[0,1]$ (or range in an interval of length at most 1).

Then we can define our utility function as the sum $f=f_A+f_B$ , so

This ensures that all outcomes with $P\left(x\right)$ are at least as good as all outcomes with $Q\left(x\right)$, without being Pascalian/fanatical to maximize $f_A$ regardless of what happens to $f_B$. Note, however, that $f_B$ may be increasingly difficult to change as the number of moral patients increases, so we may approximate Pascalian fanaticism in this limit, anyway.

For example, $f_A\left(x\right)\le -1$ if there is any suffering in $x$ that meets a certain threshold of intensity, $Q\left(x\right)$, and $f_A\left(x\right)=0$ if there is no suffering at all in $x$, $P\left(x\right)$. $f$ can still be continuous this way.

If the probability that this threshold is met is $p,0\le p<1$ and the expected value of $f_A$ conditional on this is bounded below by $-L$, $L>0$, regardless of $p$ for the choices available to you, then increasing $f_B$ by at least $pL$, which can be small, is better than trying to reduce $p$.

As another example, an AI could be incentivized to ensure it gets monitored by law enforcement. Its reward function could look like

where $I_{M_i}\left(x\right)$ is 1 if the AI is monitored by law enforcement and passes some test (or did nothing?) in period $i$, and 0 otherwise. You could put an upper bound on the number of periods or use discounting to ensure the right term can't evaluate to infinity since that would allow $f_B$ to be ignored (maybe the AI will predict its expected lifetime to be infinite), but this would eventually allow $f_B$ to overcome the $I_{M_i}$, unless you also discount the future in $f_B$.

This should also allow us to modify the utility function $f_B$, if preventing the modification would cause a test to be failed.

Furthermore, satisfying the $I_{M_i}\left(x\right)$ strongly lexically dominates increasing $f_B\left(x\right)$, but we can still make expected tradeoffs between them.

The problem then reduces to designing the AI in such a way that it can't cheat on the test, which might be something we can hard-code into it (e.g. its internal states and outputs are automatically sent to law enforcement), and so could be easier than getting $f_B$ right.

This overall approach can be repeated for any finite number of functions, $f_1,f_2,\dots ,f_n$. Recursively, you could define

for $\sigma :R\to R$ increasing and bounded with range in an interval of length at most 1, e.g. some sigmoid function. In this way, each $f_k$ dominates the previous ones, as above.

I think EA hasn't sufficiently explored the use of different types of empirical studies from which we can rigorously estimate causal effects, other than randomized controlled trials (or other experiments). This leaves us either relying heavily on subjective estimates of the magnitudes of causal effects based on weak evidence, anecdotes, expert opinion or basically guesses, or being skeptical of interventions whose cost-effectiveness estimates don't come from RCTs. I'd say I'm pretty skeptical, but not so skeptical that I think we need RCTs to conclude anything about the magnitudes of causal effects. There are methods to do causal inference from observational data.

I think this has lead us to:

1. Underexploring the global health and development space. See John Halstead's and Hauke Hillebrandt's "Growth and the case against randomista development". I think GiveWell is starting to look beyond RCTs. There's probably already a lot of research out there they can look to.

2. Relying too much on guesses and poor studies in the effective animal advocacy space (especially in the past), for example overestimating the value of leafletting. I think things have improved a lot since then, and I thought the evidence presented in the work of Rethink Priorities, Charity Entrepreneurship and Founders Pledge on corporate campaigns was good enough to meet the bar for me to donate to support corporate campaigns specifically. Humane League Labs and some academics have done and are doing research to estimate causal effects from observational data that can inform EAA.

Fehige defends the asymmetry between preference satisfaction and frustration on rationality grounds. I start from a "preference-affecting view" in this comment, and in replies, describe how to get to antifrustrationism and argue against a symmetric view.

Let's consider a given preference from the point of view of a given outcome after choosing it, in which the preference either exists or does not, by cases:

1. The preference exists:

a. If there's an outcome in which the preference exists and is more satisfied, and all else is equal, it would have been irrational to have chosen this one (over it, and at all).

b. If there's an outcome in which the preference exists and is less satisfied, and all else is equal, it would have been irrational to have chosen the other outcome (over this one, and at all).

c. If there's an outcome in which the preference does not exist, and all else is equal, the preference itself does not tell us if either would have been irrational to have chosen.

2. The preference doesn't exist:

a. If there's an outcome in which the preference exists, regardless of its degree of satisfaction, and all else equal, the preference itself does not tell us if either would have been irrational to have chosen.

So, all else equal besides the existence or degree of satisfaction of the given preference, it's always rational to choose an outcome in which the preference does not exist, but it's irrational to choose an outcome in which the preference exists but is less satisfied than in another outcome.

(I made a similar argument in the thread starting here.)

This is an argument against hedonic utility being cardinal and for widespread commensurability between hedonic experiences of different kinds. It seems that our tradeoffs, however we arrive at them, don't track the moral value of hedonic experiences.

Let X be some method or system by which we think we can establish the cardinality and/or commensurability of our hedonic experiences, and rough tradeoff rates. For example, X=reinforcement learning system in our brains, our actual choices, or our judgements of value (including intensity).

If X is not identical to our hedonic experiences, then it may be the case that X is itself what's forcing the observed cardinality and/or commensurability onto our hedonic experiences. But if it's X that's doing this, and it's the hedonic experiences themselves that are of moral value, then that cardinality and/or commensurability are properties of X, not our hedonic experiences themselves. So the observed cardinality and/or commensurability is a moral illusion.

Here's a more specific illustration of this argument:

Do our reinforcement systems have access to our whole experiences (or the whole hedonic component), or only some subsets of those neurons that are firing that are responsible for them? And what if they're more strongly connected to parts of the brain for certain kinds of experiences than others? It seems like there's a continuum of ways our reinforcement systems could be off or even badly off, so it would be more surprising to me that it would track true moral tradeoffs perfectly. Change (or add or remove) one connection between a neuron in the hedonic system and one in the reinforcement system, and now the tradeoffs made will be different, without affecting the moral value of the hedonic states. If the link between hedonic intensity and reinforcement strength is so fragile, what are the chances the reinforcement system has got it exactly right in the first place? Should be 0 (assuming my model is right).

At least for similar hedonic experiences of different intensities, if they're actually cardinal, we might expect the reinforcement system to capture some continuous monotonic transformation and not a linear transformation. But then it could be applying different monotonic transformations to different kinds of hedonic experiences. So why should we trust the tradeoffs between these different kinds of hedonic experiences?

I've been thinking more lately about how I should be thinking about causal effects for cost-effectiveness estimates, in order to clarify my own skepticism of more speculative causes, especially longtermist ones, and better understand how skeptical I ought to be. Maybe I'm far too skeptical. Maybe I just haven't come across a full model for causal effects that's convincing since I haven't been specifically looking. I've been referred to this in the past, and plan to get through it, since it might provide some missing pieces for the value of research. This also came up here.

Suppose I have two random variables, $X$ and $Y$, and I want to know the causal effect of manipulating $X$ on $Y$, if any.

1. If I'm confident there's no causal relationship between the two, say due to spatial separation, I assume there is no causal effect, and $Y$ conditional on the manipulation of $X$ to take value $A$ (possibly random), $Y|do(X=A)$, is identical to $Y$, i.e. $Y|do(X=A)=Y$. (The $do$ notation is Pearl's do-calculus notation.)

2. If $X$ could affect $Y$, but I know nothing else,

a. I might assume, based on symmetry (and chaos?) for $Y$, that $Y|do(X=A)$ and $Y$ are identical in distribution, but not necessarily literally equal as random variables. They might be slightly "shuffled" or permuted versions of each other (see symmetric decreasing rearrangements for specific examples of such a permutation). The difference in expected values is still 0. This is how I think about the effects of my every day decisions, like going to the store, breathing at particular times, etc. on future populations. I might assume the same for variables that depend on $Y$.

b. Or, I might think that manipulating $X$ just injects noise into $Y$, possibly while preserving some of its statistics, e.g. the mean or median. A simple case is just adding random symmetric noise with mean and median $0$ to $Y$. However, whether or not a statistic is preserved with the extra noise might be sensitive to the scale on which $Y$ is measured. For example, if $Y$ is real-valued, and $f:R\to R$ is strictly increasing, then for the median, $med\left(f\right(Y\left)\right)=f\left(med\right(Y\left)\right)$, but the same is not necessarily true for the expected value of $Y$, or for other variables that depend on $Y$.

c. Or, I might think that manipulating $X$ makes $Y$ closer to a "default" distribution over the possible values of $Y$, often but not always uninformed or uniform. This can shift the mean, median, etc., of $Y$. For example, $Y$ could be the face of the coin I see on my desk, and $X$ could be whether I flip the coin or not, with $X$ being not by default. So, if I do flip the coin and hence manipulate $X$, this randomizes the value of $Y$, making my probability distribution for its value uniformly random instead of a known, deterministic value. You might think that some systems are the result of optimization and therefore fragile, so random interventions might return them to prior "defaults", e.g. naive systemic change or changes to ecosystems. This could be (like) regression to the mean.

I'm not sure how to balance these three possibilities generally. If I do think the effects are symmetric, I might go with a or b or some combination of them. In particular asymmetric cases, I might also combine c.

3. Suppose I have a plausible argument for how $X$ could affect $Y$ in a particular way, but no observations that can be used as suitable proxies, even very indirect, for counterfactuals with which to estimate the size of the effect. I lean towards dealing with this case as in 2, rather than just making assumptions about effect sizes without observations.

For example, someone might propose a causal path through which $X$ affects $Y$ with a missing estimate of effect size at at least one step along the path, but an argument to that this should increase the value of $Y$. It is not enough to consider only one such path, since there may be many paths from $X$ to $Y$, e.g. different considerations for how $X$ could affect $Y$, and these would need to be combined. Some could have opposite effects. By 2, those other paths, when combined with the proposed causal path, reduce the effects of $X$ on $Y$ through the proposed path. The longer the proposed path, the more unknown alternate paths.

I think this is where I am now with speculative longtermist causes. Part of this may be my ignorance of the proposed causal paths and estimates of effect sizes, since I haven't looked too deeply at the justifications for these causes, but the dampening from unknown paths also applies when the effect sizes along a path are known, which is the next case.

4. Suppose I have a causal path through some other variable $Z$, $X\to Z\to Y$, so that $X$ causes $Z$ and $Z$ causes $Y$, and I model both the effects of $X\to Z$ and $Z\to Y$, based on observations. Should I just combine the two for the effect of $X$ on $Y$? In general, not in the straightforward way. As in 3, there could be another causal path, $X\to Z^{\prime }\to Y$ (and it could be longer, instead of with just a single intermediate variable).

As in case 3, you can think of $X\to Z^{\prime }\to Y$ as dampening the effect of $X\to Z\to Y$, and with long proposed causal paths, we might expect the net effect to be small, consistently with the intuition that the predictable impacts on the far future decrease over time due to ignorance/noise and chaos, even though the actual impacts may compound due to chaos.

Maybe I'll write this up as a full post after I've thought more about it. I imagine there's been writing related to this, including in the EA and rationality communities.

If you're a consequentialist and you think

1. each individual can sometimes sacrifice some $A$ for more $B$ for themself,

2. we should be impartial, and

3. transitivity and the independence of irrelevant alternatives hold,

then it’s sometimes ethical to sacrifice $A$ from one individual for more $B$ for another. This isn't too surprising, but let's look at the argument, which is pretty simple, and discuss some examples.

$Proof.$ Consider the following three options, with two individuals, $x$ and $y$, and $a^{+}>a$ amounts of $A$, $b^{+}>b$ amounts of $B$:

i. $x:(a:A,b^{+}:B)$, $y:(a:A,b:B)$ , read as $x$ has amount $a$ of $A$ and amount $b^{+}$ of $B$, while $y$ has amount $a$ of $A$ and amount $b$ of $B$.

ii. $x:(a^{+}:A,b:B)$, $y:(a:A,b:B)$

iii. $x:(a:A,b:B)$, $y:(a^{+}:A,b:B)$

Here we have i > ii by 1 for some $a$, $a^{+}$, $b$ and $b^{+}$, and ii = iii by impartiality, so together i > iii by 3, and we sacrifice some $A$ from $y$ for some $B$ from for $x$. $QED$

Remark: I did choose the amounts of $A$ and $B$ pretty specifically in this argument to match in certain ways. With continuous personal tradeoffs between $A$ and $B$, and continuous tradeoffs between amounts of $A$ between different individuals at all base levels of $A$, I think this should force continuous tradeoffs between one individual's amount of $A$ and another's amount of $B$. We can omit the impartiality assumption in this case.

Possible examples:

• $A=$ hedonistic welfare, $B=$ some non-hedonistic values
• $A=$ experiential values, $B=$ some non-experiential values
• $A=$ absence or negative of suffering, $B=$ knowing the truth, for its own sake (not its instrumental value)
• $A=$ absence or negative of suffering, $B=$ pleasure
• $A=$ absence or negative of suffering, $B=$ anything else that could be good
• $A=$ absence or negative of intense suffering, $B=$ absence or negative of mild suffering

In particular, if you’d be willing to endure torture for some other good, you should be willing to allow others to be tortured for you to get more of that good.

I imagine people will take this either way, e.g. some will accept that it's actually okay to let some be tortured for some other kind of benefit to different people, and others will accept that nothing can compensate them for torture. I fall into the latter camp.

Others might also reject the independence of irrelevant alternatives or transitivity, or their "spirit", e.g. by individuating options to option sets. I'm pretty undecided about independence these days.

If we think

1. it's always better to improve the welfare of an existing person (or someone who would exist anyway) than to bring others into existence, all else equal, and

2. two outcomes are (comparable and) equivalent if they have the same distribution of welfare levels (but possibly different identities; this is often called Anonymity),

then not only would we reject Mere Addition (the claim that adding good lives, even those which are barely worth living but still worth living, is never bad), but the following would be true:

Given any two nonempty populations $A$ and $B$, if any individual in $B$ is worse off than any individual in $A$, then $A\cup B$ is worse than $A$. In other words, we shouldn't add to a population any individual who isn't at least as well off as the best off in the population, all else equal.

Intuitively, adding someone with worse welfare than someone who would exist anyway is equivalent to reducing the existing individual's welfare and adding someone with better welfare than them; you just swap their welfares.

More formally, suppose $a$, a member of the original population $A$ with welfare $u$, is better off than $b$, a member of the added population $B$ with welfare $v$, so $u>v$. Then consider

$A^{\prime }$ which is $A$, but has $b$ instead of $a$, with welfare $u$.

$B^{\prime }$ which is $B$, but has $a$ instead of $b$, with welfare $v$.

Then, $A$ is better than $A^{\prime }\cup B^{\prime }$ , by the first hypothesis, because the latter has all the same individuals from $A$ (and extras from $B$) with exactly the same welfare levels, except for $a$ (from $A$ and $B^{\prime }$) who is worse off with welfare $v$ (from $B^{\prime }$) instead of $u$ (from $A$). So $A\succ A^{\prime }\cup B^{\prime }$.

And $A^{\prime }\cup B^{\prime }$ is equivalent to $A\cup B$ , by the second hypothesis, because the only difference is that we've swapped the welfare levels of $a$ and $b$. So $A^{\prime }\cup B^{\prime }\simeq A\cup B$.

So, by transitivity (and the independence of irrelevant alternatives),

My current best guess on what constitutes welfare/wellbeing/value (setting aside issues of aggregation):

1. Suffering is bad in itself.

2. Pleasure doesn't matter in itself.

3. Conscious disapproval might be bad in itself. If bad, this could capture the badness of suffering, since I see suffering as affective conscious disapproval (an externalist account).

4. Conscious approval doesn't matter in itself in an absolute sense (it may matter in a relative sense, as covered by 5). Pleasure is affective conscious approval.

5. Other kinds of preferences might matter, but only comparatively (in a wide/non-identity way) when they exist in both outcomes, i.e. between a preference that's more satisfied and the same or a different preference (of the same kind?) that's less satisfied, an outcome with the more satisfied one is better than an outcome with the less satisfied one, ignoring other reasons. This is a kind of preference-affecting principle. 

Also, I lean towards experientialism on top of this, so I think the degree of satisfaction/frustration of the preference has to be experienced for it to matter.

To expand on 5, the fact that you have an unsatisfied preference doesn't mean you disapprove of the outcome, it only means another outcome in which it is satisfied is preferable, all else equal. For example, that someone would like to go to the moon doesn't necessarily make them worse off than if they didn't have that desire, all else equal. That someone with a certain kind of disability would like to live without that disability and might even trade away part of their life to do so doesn't necessarily make them worse off, all else equal. This is incompatible with the way QALYs are estimated and used.

I think this probably can't be reconciled with the independence of irrelevant alternatives in a way that I would find satisfactory, since it would either give us antifrustrationism (which 5 explicitly rejects) or allow that sometimes having a preference is better than not, all else equal. 

More here, here and here on my shortform, and in this post.