Hide table of contents

Summary

I consider a simple version of “worldview diversification”: allocating a set amount of money per cause area per year. I explain in probably too much detail how that setup leads to inconsistent relative values from year to year and from cause area to cause area. This implies that there might be Pareto improvements, i.e., moves that you could make that will result in strictly better outcomes. However, identifying those Pareto improvements wouldn’t be trivial, and would probably require more investment into estimation and cross-area comparison capabilities.1

More elaborate versions of worldview diversification are probably able to fix this particular flaw, for example by instituting trading between the different worldview—thought that trading does ultimately have to happen. However, I view those solutions as hacks, and I suspect that the problem I outline in this post is indicative of deeper problems with the overall approach of worldview diversification.

The main flaw: inconsistent relative values

This section perhaps has too much detail to arrive at a fairly intuitive point. I thought this was worth doing because I find the point that there is a possible Pareto improvement on the table a powerful argument, and I didn’t want to hand-wave it. But the reader might want to skip to the next sections after getting the gist.

Deducing bounds for relative values from revealed preferences

Suppose that you order the ex-ante values of grants in different cause areas. The areas could be global health and development, animal welfare, speculative long-termism, etc. Their values could be given in QALYs (quality-adjusted life-years), sentience-adjusted QALYs, expected reduction in existential risk, but also in some relative unit2.

For simplicity, let us just pick the case where there are two cause areas:

More undilluted shades represent more valuable grants (e.g., larger reductions per dollar: of human suffering, animal suffering or existential risk), and lighter shades represent less valuable grants. Due to diminishing marginal returns, I’ve drawn the most valuable grants as smaller, though this doesn’t particularly matter.

Now, we can augment the picture by also considering the marginal grants which didn’t get funded.

In particular, imagine that the marginal grant which didn’t get funded for cause #1 has the same size as the marginal grant that did get funded for cause #2 (this doesn’t affect the thrust of the argument, it just makes it more apparent):

Now, from this, we can deduce some bounds on relative values:

In words rather than in shades of colour, this would be:

  • Spending L1 dollars at cost-effectiveness A greens/$ is better than spending L1 dollars at cost-effectiveness B reds/$
  • Spending L2 dollars at cost-effectiveness X reds/$ is better than spending L2 dollars at cost-effectiveness Y greens/$

Or, dividing by L1 and L2,

  • A greens is better than B reds
  • X reds is better than Y reds

In colors, this would correspond to all four squares having the same size:

Giving some values, this could be:

  • 10 greens is better than 2 reds
  • 3 reds is better than 5 greens

From this we could deduce that 6 reds > 10 greens > 2 reds, or that one green is worth between 0.2 and 0.6 reds.

But now there comes a new year

But the above was for one year. Now comes another year, with its own set of grants. But we are keeping the amount we allocate to each area constant.

It’s been a less promising year for green, and a more promising year for red, . So this means that some of the stuff that wasn’t funded last year for green is funded now, and some of the stuff that was funded last year for red isn’t funded now:

Now we can do the same comparisons as the last time:

 

And when we compare them against the previous year

we notice that there is an inconsistency.

Why is the above a problem

The above is a problem not only because there is an inconsistency, but because there is a possible Pareto improvement that’s not taken: transfer funding from cause area #2 to cause #1 in the first year, and vice-versa in year #2, and you will get both more green and more red.

With this in mind, we can review some alternatives.

Review of alternatives

Keep a “moral parliament” approach, but allow for trades in funding.

Worldview diversification might stem from a moral-parliament style set of values, where one’s values aren’t best modeled as a unitary agent, but rather as a parliament of diverse agents. And yet, the Pareto improvement argument still binds. A solution might be to start with a moral parliament, but allow trades in funding from different constituents of the parliament. More generally, one might imagine that given a parliament, that parliament might choose to become a unitary agent, and adopt a fixed, pre-negotiated exchange rate between red and green.

One problem with this approach is that the relative values arrived at through negotiation will be “dangling”, i.e., they will depend on arbitrary features of the world, like each sides' negotiating position, negotiation prowess, or hard-headedness.

I suspect that between:

  1. asking oneself how much one values outcomes in different cause areas relative to each other, and then pursuing a measure of aggregate value with more or less vigor
  2. dividing one’s funding and letting the different sides negotiate it out.

it’s very possible that a funder’s preferences would be best satisfied by the first option.

Calculate and equalize relative values

Alternatively, worldview diversification can be understood as an attempt to approximate expected value given a limited ability to estimate relative values. If so, then the answer might be to notice that worldview-diversification is a fairly imperfect approximation to any kind of utilitarian/consequentialist expected value maximization, and to try to more perfectly approximate utilitarian/consequentialist expected value maximization. This would involve estimating the relative values of projects in different areas, and attempting to equalize marginal values across cause areas and across years.

Note that there is a related but slightly different question of how harcore one should be in one’s optimization. The question is related, but I think I can imagine an arrangement where one does a somewhat chill type of optimization—for example by adding deontological constraints to one’s actions, like not doing fraud—and still strive to take all possible Pareto improvements in one’s choices.

As this relates to the Effective Altruism social movement and to Open Philanthropy

Open Philanthropy is a large foundation which is implementing a scheme similar in spirit to—but probably more sophisticated than—the simple version of worldview diversification that I outlined here. Honestly, I have little insight into the inner workings of Open Philanthropy, and I wrote this post mostly because I was annoyed, and less in the expectation of having an impact.

Still, here are two quotes from this 2017 blogposts which suggests that Open Philanthopy fell prey to the problem of inconsistent relative values:

A notable outcome of the framework we’re working on is that we will no longer have a single “benchmark” for giving now vs. later, as we did in the past. Rather, each bucket of capital will have its own standards and way of assessing grants to determine whether they qualify for drawing down the capital in that bucket. For example, there might be one bucket that aims to maximize impact according to a long-termist worldview, and another that aims to maximize impact according to a near-termist worldview; each would have different benchmarks and other criteria for deciding on whether to make a given grant or save the money for later. We think this approach is a natural outcome of worldview diversification, and will help us establish more systematic benchmarks than we currently have.

 

We see a great deal of intuitive appeal in the following principle, which we’ll call the “outlier opportunities” principle:

if we see an opportunity to do a huge, and in some sense “unusual” or “outlier,” amount of good according to worldview A by sacrificing a relatively modest, and in some sense “common” or “normal,” amount of good according to worldview B, we should do so (presuming that we consider both worldview A and worldview B highly plausible and reasonable and have deep uncertainty between them).

I think that is inelegant and could be better solved through inter-cause-area relative values.

Effective Altruism is a social movement nominally about “using evidence and reason to figure out how to benefit others as much as possible, and taking action on that basis”. It is related but distinct from Open Philanthropy. Here is a recent post from a community member arguing that decisions should not be made by pursuing “expected choiceworthiness”, but rather:

The sortition model prescribes that if you have x% credence in a theory, then you follow that theory x% of cases. If you have 20% credence in Kantianism, 30% credence in virtue ethics and 50% credence in utilitarianism, you follow Kantianism 20% of the time, virtue ethics 30% of the time and utilitarianism 50% of the time. When you act according to a theory is selected randomly with the probability of selection being the same as your credence in said theory.

It seems to me that a scheme such as this also falls prey to the problem of inconsistent relative values.

People in the Effective Altruism Movement also face the problem of trading off near-term altruistic goods—like quality-adjusted life years from global health interventions—against some chance of a larger but more speculative outcome, classically avoiding human extinction. I will relegate most discussion of this point to a footnote, but in short, if you don’t have consistent relative values between long-term and short-term interventions, then there will also be untaken Pareto improvements3.

Challenges and recommendations

Throughout this point, I have assumed that we can estimate:

  • the ex-ante value of different grants and options
  • the relative values of progress across different cause areas

The problem with this is that this is currently not possible. My impression is that estimation:

  • is pretty advanced for global health and development and adjacent cause areas
  • is nascent for animal welfare and suffering
  • is very rudimentary for speculative longtermism cause areas

My recommendation here would be to invest more in relative value estimation across cause areas. Here is some past work on steps one would have to carry out to arrive at good relative estimates.

The organization that I work for—the Quantified Uncertainty Research Institute—is doing some work on this topic, and we should be putting out a few blogposts in the near future. Others like Rethink Priorities, are also working on similar or very adjacent topics, as reviewed here.

Comments22


Sorted by Click to highlight new comments since:

I don't think this is actually a problem, for roughly the reasons described here. I.e. worldview diversification can be seen as a way of aggregating the preferences of multiple agents—but this shouldn't necessarily look like maximizing any given utility function.

I also talk about this here.

I agree that things could be such that the reflective equilibrium was that, but it would really surprise me. In particular, Open Philanthropy has a sole funder. It's one guy. It would surprise me if his preference about things was such that he wanted to leave Pareto improvements on the table. Not impossible, but it feels like a stretch. Of course I'm missing a whole lot of context, so ¯\_(ツ)_/¯.

Or, in other words, if things were such as in your post, I would expect to see trade (even though you could set up things such that it doesn't happen). I would not expect to see a U-turn on criminal justice. Instead, I think the worldview diversification thing as implemented by Open Phil is more like a historical accident.

The Pareto improvements aren't about worldview diversification, though. You can see this because you have exactly the same problem under a single worldview, if you keep the amount of funding constant per year. You can solve this by letting each worldview donate to, or steal from, its own budget in other years.

I do think trade between worldviews is good in addition to that, to avoid the costs of lending/borrowing; the issue is that you need to be very careful when you're relying on the worldviews themselves to tell you how much weight to put on them. So for example, if every year the AI risk worldview gets more and more alarmed, then it might "borrow" more and more money from the factory farming worldview, with the promise to pay back whenever it starts getting less alarmed. But the whole point of doing the bucketing in the first place is so that the factory farming worldview can protect itself from the possibility of the AI risk worldview being totally wrong/unhinged, and so you can't assume that the AI risk worldview is just as likely to update down as to update upwards.

You could avoid this by having the AI risk worldview make concrete predictions about what would make it more or less alarmed. But now this is far from a "simple" scheme, it requires doing a bunch of novel intellectual work to pin down the relevant predictions.

Re the U-turn on criminal justice: no matter what scheme you use, you should expect it to change whenever you have novel thoughts about the topic. Worldview diversification isn't a substitute for thinking!

So for example, if every year the AI risk worldview gets more and more alarmed, then it might "borrow" more and more money from the factory farming worldview, with the promise to pay back whenever it starts getting less alarmed. But the whole point of doing the bucketing in the first place is so that the factory farming worldview can protect itself from the possibility of the AI risk worldview being totally wrong/unhinged, and so you can't assume that the AI risk worldview is just as likely to update down as to update upwards.

Suppose that as the AI risk worldview becomes more alarmed, you are paying more and more units of x-risk prevention (according to the AI risk worldview) for every additional farmed animal QALY (as estimated by the farmed animal worldview). I find that very unappealing.

I agree that this, and your other comment below, both describe unappealing features of the current setup. I'm just pointing out that in fact there are unappealing outcomes all over the place, and that just because the equilibrium we've landed on has some unappealing properties doesn't mean that it's the wrong equilibrium. Specifically, the more you move towards pure maximization, the more you run into these problems; and as Holden points out, I don't think you can get out of them just by saying "let's maximize correctly".

(You might say: why not a middle ground between "fixed buckets" and "pure utility maximization"? But note that having a few buckets chosen based on standard cause prioritization reasoning is already a middle ground between pure utility maximization and the mainstream approach to charity, which does way less cause prioritization.)

I think the post from Holden that you point to isn't really enough to go from "we think really hardcore estimation is perilous" to "we should do worldview diversification". Worldview diversification is fairly specific, and there are other ways that you could rein-in optimization even if you don't have worldviews, e.g., adhering to deontological constraints, reducing "intenseness", giving good salaries to employees, and so on.

You can see this because you have exactly the same problem under a single worldview, if you keep the amount of funding constant per year. You can solve this by letting each worldview donate to, or steal from, its own budget in other years.

Good point! I think my objection then is that I do want relative values to not be "dangling". Also, I do care about expected values, so there are some interesting trades where you bet all of your budget which you couldn't do just by borrowing from your future stream.

One reason to see "dangling" relative values as principled: utility functions are equivalent (i.e. produce the same preferences over actions) up to a positive affine transformation. Hence why we often use voting systems to make decisions in cases where people's preferences clash, rather than trying to extract a metric of utility which can be compared across people.

I'm not sure whether the following example does anything for you; it could be that our intuitions about what is "elegant" are just very different:

Imagine that I kill all animals except one Koala. Then if your worldview diversification which had some weight on animals, you would spend the remaining of that worldview on the Koala. But you could buy way more human QALYs or units of x-risk work per unit of animal welfare.

More generally, setting up things such that sometimes you would end up valuing e.g., a salmon as 0.01% of a human and other times as 10% of a human just seems pretty inelegant.

This is one reason why you should use normatively defined worldviews to define your buckets, not cause areas. Animal welfare interventions currently dominate on some worldviews that include nonhuman animals and assign them substantial weight. Those worldviews aren't (almost ever) going to assign an average nonhuman animal many times more average weight than the average human. The animal-inclusive worldviews also care terminally at least about as much about any human as the panda, and assuming they are impartial and maximizing in some way, the panda won't get any help from us (unless it benefits humans substantially; humans might care more about the last panda than the average human).

Each worldview (and buckets may reflect aggregates of such worldviews) has its own claim on the relative average value of humans and salmons. Those claims can change with new information, but in the usual ways, not because there are fewer salmons or because fewer salmon are helped per $ now than before.

There can be worldviews where the marginal cost-effectiveness of GiveWell interventions are close to the marginal cost-effectiveness of the best nonhuman animal-focused interventions, and small changes to the situation can shift resources from humans towards nonhumans or vice versa. I think most worldviews won't be like that, though, and will instead have large gaps in marginal cost-effectiveness (going either way).

Open Phil has worldview buckets (https://www.openphilanthropy.org/research/update-on-cause-prioritization-at-open-philanthropy/), including an animal-inclusive bucket, but no animal-only bucket. This might look like cause area buckets instead, but what they're trying to approximate are in fact worldview buckets.

including an animal-inclusive bucket, but no animal-only bucket

Good point!

I like the Property Rights Approach, which is a formalization of resource buckets to separate representatives of theories, proportional to credences in the theories they represent. The representatives can make trades with or borrow from each other based on urgency if they agree to it. I don't think it necessarily has any issues with Pareto optimality, as long as you can force Pareto optimal cooperation. That being said, I think there are still some issues with the approach that need to be worked out.

Alternatively, worldview diversification can be understood as an attempt to approximate expected value given a limited ability to estimate relative values. If so, then the answer might be to notice that worldview-diversification is a fairly imperfect approximation to any kind of utilitarian/consequentialist expected value maximization, and to try to more perfectly approximate utilitarian/consequentialist expected value maximization. This would involve estimating the relative values of projects in different areas, and attempting to equalize marginal values across cause areas and across years.

I think this would assume away one of the main theoretical challenges of moral/normative uncertainty, which is the absence of an uncontroversial common scale to use across normative theories to measure them all against. If you expect such a common scale to exist and be uncontroversial when found, it seems like you're committed to some minimal moral realism, at least about the scale. Whether or not you're committed to that, you'd still have the problem of deciding which scale to use now given multiple candidates, which is basically the same as the original problem for moral/normative uncertainty.

See discussions of "intertheoretic comparisons".

it seems like you're committed to some minimal moral realism

I don't think I have. In particular, from a moral relativist perspective, I can notice that Open Philanthropy's funding comes from one person, notice that they have some altruistic & consequentialist inclinations and then wonder about whether worldview diversification is really the best way to go about satisfying those. 

Or even simpler, I could be saying something like: "as a moral relativist with consequentialist sympathies, this is not how I would spend my billions if I had them, because I find the dangling relative values thing inelegant."

I can notice that Open Philanthropy's funding comes from one person

One person may well have multiple different parts, or subscribe to multiple different worldviews!

asking oneself how much one values outcomes in different cause areas relative to each other, and then pursuing a measure of aggregate value with more or less vigor

I think your alternative implicitly assumes that, as a single person, you can just "decide" how much you value different outcomes. Whereas in fact I think of worldview diversification as actually a pretty good approximation of the process I'd go through internally if I were asked this question.

decide

not "decide", but "introspect", or "reflect upon", or "estimate". This is in the same way that I can estimate probabilities.

You're assuming there's a unique coherent and (e.g. vNM) rational value system there to find or settle on, rather than multiple (possibly incoherent) systems to try to weigh and no uniquely best (most satisfying) way to combine them into a single coherent (vNM) rational system. That's assuming away most of the problem.

Maybe this post can help illustrate better: https://reducing-suffering.org/two-envelopes-problem-for-brain-size-and-moral-uncertainty/

FWIW, I also find estimating unique/precise probabilities objectionably unjustifiable for similar reasons, although less bad than assuming away the hard problem of moral uncertainty.

Fair.

(I still stand by the rest of my comment, and that you're assuming away some moral/normative uncertainty, and maybe most of the problem.)

Alternatively, worldview diversification can be understood as an attempt to approximate expected value given a limited ability to estimate relative values. If so, then the answer might be to notice that worldview-diversification is a fairly imperfect approximation to any kind of utilitarian/consequentialist expected value maximization, and to try to more perfectly approximate utilitarian/consequentialist expected value maximization. This would involve estimating the relative values of projects in different areas, and attempting to equalize marginal values across cause areas and across years.

I think this is the interpretation of worldview diversification that makes the most sense, and I see two major difficulties with your alternative:

  1. The total funding amounts are not constant over time, so it doesn't make sense to equalize marginal values across time. In a year where funding decreases (e.g. FTX) all marginal values should be higher than in the previous year. And since there's no clearcut relationship between total funding and marginal values, I don't see any way to set the marginal values to be consistent across years, at least not with the information we have.

  2. More importantly, a relatively fixed funding pool (or at least a non-shrinking one) is kind of a requirement for a healthy ecosystem of projects and organizations. If the animal welfare funding bucket bounced between $100 million and $20 million and $300 million year to year, it would be really hard to actually get any animal welfare organizations up and running for an extended time.

Re: 1: Yes, the post considered a situation where funding streams are known, for simplicity. But when they are unknown, you could do expected marginal relative values. This would require some forecasting infrastructure which doesn't exist. But the solution in that case still doesn't seem like it would be worldview diversification.

Re: 2: Yes, I agree. I think you could deal with this in various ways, e.g., calculating the expected relative values of committed vs floating funding, or doing the rebalancing every few years instead of all the time. But I also think that there is some meaningful difference between a) the underlying framework, and b) how you choose to go about it, and I think this post was more about a)

I notice myself being confused about why trades have to happen at the OpenPhil level. I think Pareto Optimality in trades works best when there are more actors aggregating and talking. It's sad that donor lotteries have died out to an extent and so much regranting discourse is around internal EA social dynamics rather than impact in and of itself. 

Donor lotteries have died out? RCG got its seed funding from a donor lottery last December.

Curated and popular this week
 ·  · 11m read
 · 
Does a food carbon tax increase animal deaths and/or the total time of suffering of cows, pigs, chickens, and fish? Theoretically, this is possible, as a carbon tax could lead consumers to substitute, for example, beef with chicken. However, this is not per se the case, as animal products are not perfect substitutes.  I'm presenting the results of my master's thesis in Environmental Economics, which I re-worked and published on SSRN as a pre-print. My thesis develops a model of animal product substitution after a carbon tax, slaughter tax, and a meat tax. When I calibrate[1] this model for the U.S., there is a decrease in animal deaths and duration of suffering following a carbon tax. This suggests that a carbon tax can reduce animal suffering. Key points * Some animal products are carbon-intensive, like beef, but causes relatively few animal deaths or total time of suffering because the animals are large. Other animal products, like chicken, causes relatively many animal deaths or total time of suffering because the animals are small, but cause relatively low greenhouse gas emissions. * A carbon tax will make some animal products, like beef, much more expensive. As a result, people may buy more chicken. This would increase animal suffering, assuming that farm animals suffer. However, this is not per se the case. It is also possible that the direct negative effect of a carbon tax on chicken consumption is stronger than the indirect (positive) substitution effect from carbon-intensive products to chicken. * I developed a non-linear market model to predict the consumption of different animal products after a tax, based on own-price and cross-price elasticities. * When calibrated for the United States, this model predicts a decrease in the consumption of all animal products considered (beef, chicken, pork, and farmed fish). Therefore, the modelled carbon tax is actually good for animal welfare, assuming that animals live net-negative lives. * A slaughter tax (a
MarieF🔸
 ·  · 4m read
 · 
Summary * After >2 years at Hi-Med, I have decided to step down from my role. * This allows me to complete my medical residency for long-term career resilience, whilst still allowing part-time flexibility for direct charity work. It also allows me to donate more again. * Hi-Med is now looking to appoint its next Executive Director; the application deadline is 26 January 2025. * I will join Hi-Med’s governing board once we have appointed the next Executive Director. Before the role When I graduated from medical school in 2017, I had already started to give 10% of my income to effective charities, but I was unsure as to how I could best use my medical degree to make this world a better place. After dipping my toe into nonprofit fundraising (with Doctors Without Borders) and working in a medical career-related start-up to upskill, a talk given by Dixon Chibanda at EAG London 2018 deeply inspired me. I formed a rough plan to later found an organisation that would teach Post-traumatic stress disorder (PTSD)-specific psychotherapeutic techniques to lay people to make evidence-based treatment of PTSD scalable. I started my medical residency in psychosomatic medicine in 2019, working for a specialised clinic for PTSD treatment until 2021, then rotated to child and adolescent psychiatry for a year and was half a year into the continuation of my specialisation training at a third hospital, when Akhil Bansal, whom I met at a recent EAG in London, reached out and encouraged me to apply for the ED position at Hi-Med - an organisation that I knew through my participation in their introductory fellowship (an academic paper about the outcomes of this first cohort can be found here). I seized the opportunity, applied, was offered the position, and started working full-time in November 2022.  During the role I feel truly privileged to have had the opportunity to lead High Impact Medicine for the past two years. My learning curve was steep - there were so many new things to
Ozzie Gooen
 ·  · 9m read
 · 
We’re releasing Squiggle AI, a tool that generates probabilistic models using the Squiggle language. This can provide early cost-effectiveness models and other kinds of probabilistic programs. No prior Squiggle knowledge is required to use Squiggle AI. Simply ask for whatever you want to estimate, and the results should be fairly understandable. The Squiggle programming language acts as an adjustable backend, but isn’t mandatory to learn. Beyond being directly useful, we’re interested in Squiggle AI as an experiment in epistemic reasoning with LLMs. We hope it will help highlight potential strengths, weaknesses, and directions for the field. Screenshots The “Playground” view after it finishes a successful workflow. Form on the left, code in the middle, code output on the right.The “Steps” page. Shows all of the steps that the workflow went through, next to the form on the left. For each, shows a simplified view of recent messages to and from the LLM. Motivation Organizations in the effective altruism and rationalist communities regularly rely on cost-effectiveness analyses and fermi estimates to guide their decisions. QURI's mission is to make these probabilistic tools more accessible and reliable for altruistic causes. However, our experience with tools like Squiggle and Guesstimate has revealed a significant challenge: even highly skilled domain experts frequently struggle with the basic programming requirements and often make errors in their models. This suggests a need for alternative approaches. Language models seem particularly well-suited to address these difficulties. Fermi estimates typically follow straightforward patterns and rely on common assumptions, making them potentially ideal candidates for LLM assistance. Previous direct experiments with Claude and ChatGPT alone proved insufficient, but with substantial iteration, we've developed a framework that significantly improves the output quality and user experience. We're focusing specifically on