In this post, I will apply two methods used in dealing with moral uncertainty to a different problem, which is the one of choosing between radically divergent worldviews.

Imagine a young and altruistically motivated graduate. As she learns more, her circle of moral concern expands from people near her to all humans, and then to animals. At some point, she encounters arguments for prioritising the long-term future. Suddenly, all her previous efforts seem futile and it takes her mere seconds to decide that she wants to put her life in service of creating a good future. Just as she embarks on pursuing her completely re-ordered priorities, she halts for a second to contemplate the train of thought that led her thus far. She knows that not many people share her new priorities, and that most have radically different worldviews from hers. How certain can she be that she has not made a mistake on the way?

The above is an example of the kind of question discussed in the 80.000 hours podcast with Holden Karnofsky. We can roughly formulate the question as the one of how to split resources between worldviews that recommend very different courses of action. This kind of uncertainty, not about the claims of a certain theory, but about which of these theories we should choose, is similar to problems around moral uncertainty, like the ones discussed in Will MacAskill's thesis. What I will be doing in the following is to apply two of the methods used in that thesis to a concrete example of someone trying to figure out which actions to prioritise, given uncertainty as to which worldview is correct.

Worldviews, and where they diverge

One example that Holden discusses in the podcast is the comparison between helping humans and animals (alive today). So, for instance, we could help a human for $1,000 and a chicken for $1 and then face the decision which one to prioritise.

This comes down to questions of whether we value chickens, on what grounds we would value humans and not chickens, and how much exactly we should value chickens if we do value them. So, if you value chickens 10% as much as humans, you might want to decide in favour of chickens, and if you do not value chickens at all, helping the humans would be a much better decision.

We can think of these two outlooks as two different "worldviews". While I do not have a clear definition of that term (Holden himself says that the concept is "a very fuzzy idea"), relevant features of worldviews are:

• Each yields particular outcomes (e.g. "prioritise X") that are very different from those of other worldviews.
• These different outcomes might hinge on answering just one question differently (e.g. "Do chickens have moral value?"), or perhaps a few crucial questions.
• Worldviews are constituted by a combination of views and beliefs that are not only normative, but also empirical, which makes it difficult to disentangle claims they make when we want to deal with uncertainty about worldviews (thanks to Max Daniel for pointing this out).

In the next section, I will discuss moral uncertainty, that is, uncertainties between first-order normative theories (such as Virtue Ethics, Hedonistic Utilitarianism, or Kantianism). This does not mean that I think that worldviews and normative theories are the same thing, even though I think we can apply measures of uncertainty to them in similar ways. I will also use the more neutral term "view", whenever I do not think it is important to distinguish the two terms.

The worldview split problem on an individual basis

Let's assume the fictitious graduate mentioned above (let's call her Lisa) has to pick a single option, for example in deciding which cause to devote her career to. When making this choice, it seems reasonable to not go for the option she likes most, but to also take into account how certain she is that each of the worldviews she derives the options from is correct.

This kind of thinking is what motivates the methods which Will uses in his thesis on normative uncertainty. He discusses examples in which agents have to weigh options which are ordered differently according to different normative theories, while being uncertain as to which of the theories is correct.

These decision-situations include a decision-maker at a specific point in time, who has at her disposal

• a set of possible options (actions she can perform),
• a set of theories that give an ordering of the options in terms of their choice-worthiness (whether, and sometimes, how much an option is preferred),
• and a credence function that assigns each theory a subjective likelihood of being true.

This process is analogous to expected utility calculation, replacing "expected utility" with "expected choice-worthiness".

Below, I will apply these methods - ways of calculating and maximising expected choice-worthiness - to the case of different worldviews and see whether it is helpful in dealing with the kinds of "worldview split problems" discussed above.

Finding theories behind the worldviews

One major difficulty in assigning credences to worldviews is that often it is not clear what exactly constitutes a worldview. In this section, I explain why in the comparison further down, I am using the distinction between person-affecting and impersonal views to rank options. Feel free to skip to the next section if you don't want to hear all the details!

Person-affecting views capture the intuition that an act can be bad only if it is bad for someone, while impersonal views aim to maximize total (net) wellbeing.

The reason I searched for a different split in worldviews is that just attributing one worldview to each option made the whole prioritising framework uninformative: if our worldview is "I should care most about humans alive today" and one option is "do whatever is best for humans alive today", the framework's results will always be aligned with the worldview we assign highest credence to. If we assign this credence mainly based on intuition, we end up with what we would do if we just went for whatever we like best.

What I did to get around this was to note down some things someone might plausibly care about, and try to cluster them into more informative categories.

Things Lisa cares about:

• Pursuing paths of action that produce good consequences.

• Trying to cultivate care for others. This works well for other humans, somewhat for animals, but breaks down when we think about alien minds.

• Capability to suffer as a very important aspect in who we assign moral value to.

• Self-improvement: this includes virtues of good conduct in interaction with others, virtues like wisdom aimed at improving the way one makes decisions, and a general striving to become a better person. 

In addition, a flourishing world in which beings are happy, satisfy their preferences, and realise a variety of values seems like something worth caring about.

A lot of these values can be made use of in different worldviews: a commitment to bringing about the best possible consequences will vary according to which consequences are judged as best. A commitment to cultivating care will vary according to which beings we think of as ultimately "care-worthy" (other humans? animals? future beings?). For both, the person-affecting/impersonal distinction is relevant in deciding in which beings we should care about and which consequences we judge to be good. Capability to suffer seems like a less volatile criterion, which is uncertain still, but I'm hoping that this uncertainty is mainly empirical.

Looking at different virtues, there are some which which seem to favour a presentist perspective, centering around improving oneself and caring for others in the moment, and others which might favour long-termism (e.g. if we think that humanity as a whole is acting wisely by preventing its own extinction).

I have tried to show that just caring about producing good consequences or being virtuous does not give us an informative ranking of options, but that distinguishing between person-affecting and impersonal views does. I will apply this idea in the next section.

Applying this

Borda Rule

I started by ranking the options according to the Borda Rule, which Will uses in his thesis for theories that give merely ordinal choice-worthiness (that is, they give a preferred ranking, but do not provide information as to how much better or worse an option is) and where choice-worthiness is not comparable across theories.

The above table gives an ordering of preferences which seems roughly plausible to me, but might vary according to whether we take "person-affecting" to be about humans or including animals, and how we deal with the future. For example, we might think only about persons that already exist today, or additionally consider people who would have come into existence regardless of our intervention.

Multiplying the Borda score with the credence Lisa assigns to person-affecting and impersonal views, we get the following:

(A): 0.4*2+0.6*0= 0.8

(B): 0.4*1+0.6*1= 1

(C): 0.4*0+0.6*2= 1.2

We see that under this calculation, Option (C) - prioritizing the long-term future - scores highest. If we are very uncertain about which view is best and assign a 50% credence to each, we get:

(A): 0.5*2+0.5*0= 1

(B): 0.5*1+0.5*1= 1

(C): 0.5*0+0.5*2= 1

This makes it seem like, in this case, a merely ordinal ranking is still not better than just going with a simple intuition. If we add the assumption that choice-worthiness can be compared across different theories, and we think we can say something about how much more desirable one outcome is over the other, we can try our luck with cardinal ordering and see whether that is more informative.

Cardinal Ordering

For example, we might assign the following choice-worthiness to the different options under the two theories we are considering:

Credence-weighted, we get the following results across the two views:

(A) = 0.4*0.7+0.6*0.05= 0.31

(B) = 0.4*0.2+0.6*0.05= 0.11

(C) = 0.4*0.1+0.6*0.9= 0.58

Where do I get these numbers from? They are based on my intuition that under person-affecting views, humans alive today are considered as more important than either animals and the long-term future, while for impersonal views, the long-term future outweighs both current human and animal welfare by a lot. I was surprised to see such a low score assigned to animal welfare, since I intuitively think that animal welfare matters a lot. What the low score reflects is my (until now implicit!) assumption that person-affecting views really only deal with persons, excluding non-human animals. It also shows that I think person-affecting views only care about persons alive today, which is a strand of person-affecting views that differs from "caring only about people who would have existed anyway".

If we do think that there is some merit in the intuitive cardinal ordering I provided, this ordering still gives the same result for a 50-50 credence:

(A) = 0.5*0.7+0.5*0.05= 0.375

(B) = 0.5*0.2+0.5*0.05= 0.125

(C) = 0.5*0.1+0.5*0.9= 0.5

Option A would only be favoured if we assign 60% credence or more to person-affecting views being right (yielding a score of 0.44 versus option C scoring 0.42). This means that if Lisa gathered evidence that made her assign a credence of 60% or more to person-affecting views, she would have to not prioritise working on the long-term future, but on helping humans alive today.

Problems

I am aware of the assigned credences seeming somewhat random, in only reflecting the subjective likelihood that someone assigns to a certain view being true. I am also unsure about whether the orderings I ascribed to different views are correct interpretations of these views. However, I hope that having ascribed numbers to different orderings might facilitate discussion later on, making assumptions more tangible and thereby easier to challenge. The extent to which we should be bothered about this problem may also depend on what we would hope to use this method for.

As I mentioned above, worldviews encompass lots of different claims which are hard to disentangle. I tried to get around this by turning this into a question about person-affecting versus impersonal views, but very likely lost something along the way. This is why I hope that someone else will find this a useful prompt to propose a better way of thinking about this question.

Conclusion

Was this useful? /What did I learn? / What will I do as a result?

I am probably biased in favour of thinking that writing this post was useful because I invested time in doing so. Other reasons why this felt like a useful pursuit are that it was much more productive than just spending time reading on my own, since setting myself a deadline increased the amount I read and having a question in mind made my reading more focused and engaging.

As for other people, I hope this might provide a model that others can use to make explicit their thinking about their priorities and their uncertainties therein. Other benefits might lie in motivating people to engage with the question at large, or in prompting someone to see flaws in what I was doing and do it better in the future.

I learnt what my intuitive models of different moral theories look like and now have a much clearer grasp of where they are not as clear as I would hope them to be. I also enjoyed understanding what information might change my personal cause prioritisation. One immediate result of writing this is that I have a much better sense of which things I want to learn more about because doing so might affect my future behaviour.

If you have any answers to the questions and problems raised above, or anything else you would like to add, please do leave a comment!