Hide table of contents

Summary

I consider a simple version of “worldview diversification”: allocating a set amount of money per cause area per year. I explain in probably too much detail how that setup leads to inconsistent relative values from year to year and from cause area to cause area. This implies that there might be Pareto improvements, i.e., moves that you could make that will result in strictly better outcomes. However, identifying those Pareto improvements wouldn’t be trivial, and would probably require more investment into estimation and cross-area comparison capabilities.1

More elaborate versions of worldview diversification are probably able to fix this particular flaw, for example by instituting trading between the different worldview—thought that trading does ultimately have to happen. However, I view those solutions as hacks, and I suspect that the problem I outline in this post is indicative of deeper problems with the overall approach of worldview diversification.

The main flaw: inconsistent relative values

This section perhaps has too much detail to arrive at a fairly intuitive point. I thought this was worth doing because I find the point that there is a possible Pareto improvement on the table a powerful argument, and I didn’t want to hand-wave it. But the reader might want to skip to the next sections after getting the gist.

Deducing bounds for relative values from revealed preferences

Suppose that you order the ex-ante values of grants in different cause areas. The areas could be global health and development, animal welfare, speculative long-termism, etc. Their values could be given in QALYs (quality-adjusted life-years), sentience-adjusted QALYs, expected reduction in existential risk, but also in some relative unit2.

For simplicity, let us just pick the case where there are two cause areas:

More undilluted shades represent more valuable grants (e.g., larger reductions per dollar: of human suffering, animal suffering or existential risk), and lighter shades represent less valuable grants. Due to diminishing marginal returns, I’ve drawn the most valuable grants as smaller, though this doesn’t particularly matter.

Now, we can augment the picture by also considering the marginal grants which didn’t get funded.

In particular, imagine that the marginal grant which didn’t get funded for cause #1 has the same size as the marginal grant that did get funded for cause #2 (this doesn’t affect the thrust of the argument, it just makes it more apparent):

Now, from this, we can deduce some bounds on relative values:

In words rather than in shades of colour, this would be:

  • Spending L1 dollars at cost-effectiveness A greens/$ is better than spending L1 dollars at cost-effectiveness B reds/$
  • Spending L2 dollars at cost-effectiveness X reds/$ is better than spending L2 dollars at cost-effectiveness Y greens/$

Or, dividing by L1 and L2,

  • A greens is better than B reds
  • X reds is better than Y reds

In colors, this would correspond to all four squares having the same size:

Giving some values, this could be:

  • 10 greens is better than 2 reds
  • 3 reds is better than 5 greens

From this we could deduce that 6 reds > 10 greens > 2 reds, or that one green is worth between 0.2 and 0.6 reds.

But now there comes a new year

But the above was for one year. Now comes another year, with its own set of grants. But we are keeping the amount we allocate to each area constant.

It’s been a less promising year for green, and a more promising year for red, . So this means that some of the stuff that wasn’t funded last year for green is funded now, and some of the stuff that was funded last year for red isn’t funded now:

Now we can do the same comparisons as the last time:

 

And when we compare them against the previous year

we notice that there is an inconsistency.

Why is the above a problem

The above is a problem not only because there is an inconsistency, but because there is a possible Pareto improvement that’s not taken: transfer funding from cause area #2 to cause #1 in the first year, and vice-versa in year #2, and you will get both more green and more red.

With this in mind, we can review some alternatives.

Review of alternatives

Keep a “moral parliament” approach, but allow for trades in funding.

Worldview diversification might stem from a moral-parliament style set of values, where one’s values aren’t best modeled as a unitary agent, but rather as a parliament of diverse agents. And yet, the Pareto improvement argument still binds. A solution might be to start with a moral parliament, but allow trades in funding from different constituents of the parliament. More generally, one might imagine that given a parliament, that parliament might choose to become a unitary agent, and adopt a fixed, pre-negotiated exchange rate between red and green.

One problem with this approach is that the relative values arrived at through negotiation will be “dangling”, i.e., they will depend on arbitrary features of the world, like each sides' negotiating position, negotiation prowess, or hard-headedness.

I suspect that between:

  1. asking oneself how much one values outcomes in different cause areas relative to each other, and then pursuing a measure of aggregate value with more or less vigor
  2. dividing one’s funding and letting the different sides negotiate it out.

it’s very possible that a funder’s preferences would be best satisfied by the first option.

Calculate and equalize relative values

Alternatively, worldview diversification can be understood as an attempt to approximate expected value given a limited ability to estimate relative values. If so, then the answer might be to notice that worldview-diversification is a fairly imperfect approximation to any kind of utilitarian/consequentialist expected value maximization, and to try to more perfectly approximate utilitarian/consequentialist expected value maximization. This would involve estimating the relative values of projects in different areas, and attempting to equalize marginal values across cause areas and across years.

Note that there is a related but slightly different question of how harcore one should be in one’s optimization. The question is related, but I think I can imagine an arrangement where one does a somewhat chill type of optimization—for example by adding deontological constraints to one’s actions, like not doing fraud—and still strive to take all possible Pareto improvements in one’s choices.

As this relates to the Effective Altruism social movement and to Open Philanthropy

Open Philanthropy is a large foundation which is implementing a scheme similar in spirit to—but probably more sophisticated than—the simple version of worldview diversification that I outlined here. Honestly, I have little insight into the inner workings of Open Philanthropy, and I wrote this post mostly because I was annoyed, and less in the expectation of having an impact.

Still, here are two quotes from this 2017 blogposts which suggests that Open Philanthopy fell prey to the problem of inconsistent relative values:

A notable outcome of the framework we’re working on is that we will no longer have a single “benchmark” for giving now vs. later, as we did in the past. Rather, each bucket of capital will have its own standards and way of assessing grants to determine whether they qualify for drawing down the capital in that bucket. For example, there might be one bucket that aims to maximize impact according to a long-termist worldview, and another that aims to maximize impact according to a near-termist worldview; each would have different benchmarks and other criteria for deciding on whether to make a given grant or save the money for later. We think this approach is a natural outcome of worldview diversification, and will help us establish more systematic benchmarks than we currently have.

 

We see a great deal of intuitive appeal in the following principle, which we’ll call the “outlier opportunities” principle:

if we see an opportunity to do a huge, and in some sense “unusual” or “outlier,” amount of good according to worldview A by sacrificing a relatively modest, and in some sense “common” or “normal,” amount of good according to worldview B, we should do so (presuming that we consider both worldview A and worldview B highly plausible and reasonable and have deep uncertainty between them).

I think that is inelegant and could be better solved through inter-cause-area relative values.

Effective Altruism is a social movement nominally about “using evidence and reason to figure out how to benefit others as much as possible, and taking action on that basis”. It is related but distinct from Open Philanthropy. Here is a recent post from a community member arguing that decisions should not be made by pursuing “expected choiceworthiness”, but rather:

The sortition model prescribes that if you have x% credence in a theory, then you follow that theory x% of cases. If you have 20% credence in Kantianism, 30% credence in virtue ethics and 50% credence in utilitarianism, you follow Kantianism 20% of the time, virtue ethics 30% of the time and utilitarianism 50% of the time. When you act according to a theory is selected randomly with the probability of selection being the same as your credence in said theory.

It seems to me that a scheme such as this also falls prey to the problem of inconsistent relative values.

People in the Effective Altruism Movement also face the problem of trading off near-term altruistic goods—like quality-adjusted life years from global health interventions—against some chance of a larger but more speculative outcome, classically avoiding human extinction. I will relegate most discussion of this point to a footnote, but in short, if you don’t have consistent relative values between long-term and short-term interventions, then there will also be untaken Pareto improvements3.

Challenges and recommendations

Throughout this point, I have assumed that we can estimate:

  • the ex-ante value of different grants and options
  • the relative values of progress across different cause areas

The problem with this is that this is currently not possible. My impression is that estimation:

  • is pretty advanced for global health and development and adjacent cause areas
  • is nascent for animal welfare and suffering
  • is very rudimentary for speculative longtermism cause areas

My recommendation here would be to invest more in relative value estimation across cause areas. Here is some past work on steps one would have to carry out to arrive at good relative estimates.

The organization that I work for—the Quantified Uncertainty Research Institute—is doing some work on this topic, and we should be putting out a few blogposts in the near future. Others like Rethink Priorities, are also working on similar or very adjacent topics, as reviewed here.

45

0
0

Reactions

0
0

More posts like this

Comments22
Sorted by Click to highlight new comments since: Today at 10:27 AM

I don't think this is actually a problem, for roughly the reasons described here. I.e. worldview diversification can be seen as a way of aggregating the preferences of multiple agents—but this shouldn't necessarily look like maximizing any given utility function.

I also talk about this here.

I agree that things could be such that the reflective equilibrium was that, but it would really surprise me. In particular, Open Philanthropy has a sole funder. It's one guy. It would surprise me if his preference about things was such that he wanted to leave Pareto improvements on the table. Not impossible, but it feels like a stretch. Of course I'm missing a whole lot of context, so ¯\_(ツ)_/¯.

Or, in other words, if things were such as in your post, I would expect to see trade (even though you could set up things such that it doesn't happen). I would not expect to see a U-turn on criminal justice. Instead, I think the worldview diversification thing as implemented by Open Phil is more like a historical accident.

The Pareto improvements aren't about worldview diversification, though. You can see this because you have exactly the same problem under a single worldview, if you keep the amount of funding constant per year. You can solve this by letting each worldview donate to, or steal from, its own budget in other years.

I do think trade between worldviews is good in addition to that, to avoid the costs of lending/borrowing; the issue is that you need to be very careful when you're relying on the worldviews themselves to tell you how much weight to put on them. So for example, if every year the AI risk worldview gets more and more alarmed, then it might "borrow" more and more money from the factory farming worldview, with the promise to pay back whenever it starts getting less alarmed. But the whole point of doing the bucketing in the first place is so that the factory farming worldview can protect itself from the possibility of the AI risk worldview being totally wrong/unhinged, and so you can't assume that the AI risk worldview is just as likely to update down as to update upwards.

You could avoid this by having the AI risk worldview make concrete predictions about what would make it more or less alarmed. But now this is far from a "simple" scheme, it requires doing a bunch of novel intellectual work to pin down the relevant predictions.

Re the U-turn on criminal justice: no matter what scheme you use, you should expect it to change whenever you have novel thoughts about the topic. Worldview diversification isn't a substitute for thinking!

So for example, if every year the AI risk worldview gets more and more alarmed, then it might "borrow" more and more money from the factory farming worldview, with the promise to pay back whenever it starts getting less alarmed. But the whole point of doing the bucketing in the first place is so that the factory farming worldview can protect itself from the possibility of the AI risk worldview being totally wrong/unhinged, and so you can't assume that the AI risk worldview is just as likely to update down as to update upwards.

Suppose that as the AI risk worldview becomes more alarmed, you are paying more and more units of x-risk prevention (according to the AI risk worldview) for every additional farmed animal QALY (as estimated by the farmed animal worldview). I find that very unappealing.

I agree that this, and your other comment below, both describe unappealing features of the current setup. I'm just pointing out that in fact there are unappealing outcomes all over the place, and that just because the equilibrium we've landed on has some unappealing properties doesn't mean that it's the wrong equilibrium. Specifically, the more you move towards pure maximization, the more you run into these problems; and as Holden points out, I don't think you can get out of them just by saying "let's maximize correctly".

(You might say: why not a middle ground between "fixed buckets" and "pure utility maximization"? But note that having a few buckets chosen based on standard cause prioritization reasoning is already a middle ground between pure utility maximization and the mainstream approach to charity, which does way less cause prioritization.)

I think the post from Holden that you point to isn't really enough to go from "we think really hardcore estimation is perilous" to "we should do worldview diversification". Worldview diversification is fairly specific, and there are other ways that you could rein-in optimization even if you don't have worldviews, e.g., adhering to deontological constraints, reducing "intenseness", giving good salaries to employees, and so on.

You can see this because you have exactly the same problem under a single worldview, if you keep the amount of funding constant per year. You can solve this by letting each worldview donate to, or steal from, its own budget in other years.

Good point! I think my objection then is that I do want relative values to not be "dangling". Also, I do care about expected values, so there are some interesting trades where you bet all of your budget which you couldn't do just by borrowing from your future stream.

One reason to see "dangling" relative values as principled: utility functions are equivalent (i.e. produce the same preferences over actions) up to a positive affine transformation. Hence why we often use voting systems to make decisions in cases where people's preferences clash, rather than trying to extract a metric of utility which can be compared across people.

I'm not sure whether the following example does anything for you; it could be that our intuitions about what is "elegant" are just very different:

Imagine that I kill all animals except one Koala. Then if your worldview diversification which had some weight on animals, you would spend the remaining of that worldview on the Koala. But you could buy way more human QALYs or units of x-risk work per unit of animal welfare.

More generally, setting up things such that sometimes you would end up valuing e.g., a salmon as 0.01% of a human and other times as 10% of a human just seems pretty inelegant.

This is one reason why you should use normatively defined worldviews to define your buckets, not cause areas. Animal welfare interventions currently dominate on some worldviews that include nonhuman animals and assign them substantial weight. Those worldviews aren't (almost ever) going to assign an average nonhuman animal many times more average weight than the average human. The animal-inclusive worldviews also care terminally at least about as much about any human as the panda, and assuming they are impartial and maximizing in some way, the panda won't get any help from us (unless it benefits humans substantially; humans might care more about the last panda than the average human).

Each worldview (and buckets may reflect aggregates of such worldviews) has its own claim on the relative average value of humans and salmons. Those claims can change with new information, but in the usual ways, not because there are fewer salmons or because fewer salmon are helped per $ now than before.

There can be worldviews where the marginal cost-effectiveness of GiveWell interventions are close to the marginal cost-effectiveness of the best nonhuman animal-focused interventions, and small changes to the situation can shift resources from humans towards nonhumans or vice versa. I think most worldviews won't be like that, though, and will instead have large gaps in marginal cost-effectiveness (going either way).

Open Phil has worldview buckets (https://www.openphilanthropy.org/research/update-on-cause-prioritization-at-open-philanthropy/), including an animal-inclusive bucket, but no animal-only bucket. This might look like cause area buckets instead, but what they're trying to approximate are in fact worldview buckets.

including an animal-inclusive bucket, but no animal-only bucket

Good point!

I like the Property Rights Approach, which is a formalization of resource buckets to separate representatives of theories, proportional to credences in the theories they represent. The representatives can make trades with or borrow from each other based on urgency if they agree to it. I don't think it necessarily has any issues with Pareto optimality, as long as you can force Pareto optimal cooperation. That being said, I think there are still some issues with the approach that need to be worked out.

Alternatively, worldview diversification can be understood as an attempt to approximate expected value given a limited ability to estimate relative values. If so, then the answer might be to notice that worldview-diversification is a fairly imperfect approximation to any kind of utilitarian/consequentialist expected value maximization, and to try to more perfectly approximate utilitarian/consequentialist expected value maximization. This would involve estimating the relative values of projects in different areas, and attempting to equalize marginal values across cause areas and across years.

I think this would assume away one of the main theoretical challenges of moral/normative uncertainty, which is the absence of an uncontroversial common scale to use across normative theories to measure them all against. If you expect such a common scale to exist and be uncontroversial when found, it seems like you're committed to some minimal moral realism, at least about the scale. Whether or not you're committed to that, you'd still have the problem of deciding which scale to use now given multiple candidates, which is basically the same as the original problem for moral/normative uncertainty.

See discussions of "intertheoretic comparisons".

it seems like you're committed to some minimal moral realism

I don't think I have. In particular, from a moral relativist perspective, I can notice that Open Philanthropy's funding comes from one person, notice that they have some altruistic & consequentialist inclinations and then wonder about whether worldview diversification is really the best way to go about satisfying those. 

Or even simpler, I could be saying something like: "as a moral relativist with consequentialist sympathies, this is not how I would spend my billions if I had them, because I find the dangling relative values thing inelegant."

I can notice that Open Philanthropy's funding comes from one person

One person may well have multiple different parts, or subscribe to multiple different worldviews!

asking oneself how much one values outcomes in different cause areas relative to each other, and then pursuing a measure of aggregate value with more or less vigor

I think your alternative implicitly assumes that, as a single person, you can just "decide" how much you value different outcomes. Whereas in fact I think of worldview diversification as actually a pretty good approximation of the process I'd go through internally if I were asked this question.

decide

not "decide", but "introspect", or "reflect upon", or "estimate". This is in the same way that I can estimate probabilities.

You're assuming there's a unique coherent and (e.g. vNM) rational value system there to find or settle on, rather than multiple (possibly incoherent) systems to try to weigh and no uniquely best (most satisfying) way to combine them into a single coherent (vNM) rational system. That's assuming away most of the problem.

Maybe this post can help illustrate better: https://reducing-suffering.org/two-envelopes-problem-for-brain-size-and-moral-uncertainty/

FWIW, I also find estimating unique/precise probabilities objectionably unjustifiable for similar reasons, although less bad than assuming away the hard problem of moral uncertainty.

Fair.

(I still stand by the rest of my comment, and that you're assuming away some moral/normative uncertainty, and maybe most of the problem.)

Alternatively, worldview diversification can be understood as an attempt to approximate expected value given a limited ability to estimate relative values. If so, then the answer might be to notice that worldview-diversification is a fairly imperfect approximation to any kind of utilitarian/consequentialist expected value maximization, and to try to more perfectly approximate utilitarian/consequentialist expected value maximization. This would involve estimating the relative values of projects in different areas, and attempting to equalize marginal values across cause areas and across years.

I think this is the interpretation of worldview diversification that makes the most sense, and I see two major difficulties with your alternative:

  1. The total funding amounts are not constant over time, so it doesn't make sense to equalize marginal values across time. In a year where funding decreases (e.g. FTX) all marginal values should be higher than in the previous year. And since there's no clearcut relationship between total funding and marginal values, I don't see any way to set the marginal values to be consistent across years, at least not with the information we have.

  2. More importantly, a relatively fixed funding pool (or at least a non-shrinking one) is kind of a requirement for a healthy ecosystem of projects and organizations. If the animal welfare funding bucket bounced between $100 million and $20 million and $300 million year to year, it would be really hard to actually get any animal welfare organizations up and running for an extended time.

Re: 1: Yes, the post considered a situation where funding streams are known, for simplicity. But when they are unknown, you could do expected marginal relative values. This would require some forecasting infrastructure which doesn't exist. But the solution in that case still doesn't seem like it would be worldview diversification.

Re: 2: Yes, I agree. I think you could deal with this in various ways, e.g., calculating the expected relative values of committed vs floating funding, or doing the rebalancing every few years instead of all the time. But I also think that there is some meaningful difference between a) the underlying framework, and b) how you choose to go about it, and I think this post was more about a)

I notice myself being confused about why trades have to happen at the OpenPhil level. I think Pareto Optimality in trades works best when there are more actors aggregating and talking. It's sad that donor lotteries have died out to an extent and so much regranting discourse is around internal EA social dynamics rather than impact in and of itself. 

Donor lotteries have died out? RCG got its seed funding from a donor lottery last December.

Curated and popular this week
Relevant opportunities