Modeling the (dis)value of human survival and expansion

Written in February 2023. I’ve only made some light style edits before posting. This means that I do not reference all related work written since then. Also, this post does not necessarily reflect my current views. I’ve decided to publish this even though I had never finished the last subsection and was unhappy with the model, but I now think some might find this useful.

The longtermist case for prioritizing risks of human extinction/collapse over trajectory changes heavily relies on what seems to be an under-questioned and under-studied assumption: the future, conditional on humanity’s survival and preserved potential^[1], will be highly positive in expectation (or the future is good, for short).^[2]

There have been some interesting discussions on the subject (see, e.g., Christiano 2013; Reese 2016; West 2017; Brauner and Grosse-Holz 2018; DiGiovanni 2021; MacAskill 2022, Chapter 9). Jacy Reese Anthis (2022) has even come up with a seemingly exhaustive list of (classes of) arguments for and against assuming the future will be good and proposed a framework for quantitatively estimating the expected value of the future. While I agree with him that using his framework is probably much better than just following our crude intuitions, I think that (i) his breakdown makes it very hard not to double-count some arguments, and (ii) we could come up with a model that weights the arguments in a much more rigorous way. This is why I’m writing this post (which builds on all the work I just referenced).

I first attempt to list all the general factors we should consider, before building a proper model, weighing the different factors against one another. I then review and comment on the limitations/assumptions of this model, before pointing to research questions that must be tackled for us to better estimate the values of its parameters.

I won’t share my own estimates of the variables (nor the result my model gives me), since I want this post to remain a tool people can use, not an aggregation of my beliefs about future trajectories. However, I’d be happy for other people to quickly use the model and share their results in the comments (or in a separate post) since I believe this forecasting question to be tremendously important and neglected.

Epistemic status: Although I’ve been thinking quite a bit about this question, forecasting the value of the future is severely underinvestigated and small as an area of interest. I wouldn’t be surprised if some comments make me/us realize I/we overlooked some non-trivial considerations and incentivize me to come up with a model 2.0 to correct that. On the other hand, I’m confident this piece is more rigorous and more informative than previous (somewhat) related essays, since this is the whole point of it and builds on those.

Read the rest of the post in this Google Doc. I couldn't easily copy-paste the math notations here.

1. ^{^}

    Potential to mitigate the “astronomical waste” (see Bostrom 2003).

2. ^{^}

    “Longtermism is often thought to lead to the conclusion that extinction risk reduction should be a global priority. This presupposes that the expected social value of continued human existence is positive. But one can imagine scenarios and social welfare criteria according to which humanity’s future should be expected to contain more bad than good. Before engaging in more fine-grained cause prioritisation across efforts to reduce extinction risk, it is therefore important to consider the sign and magnitude of the expected social value of the continued existence of humanity.” (Global Prioirities Research, 2020)

3. ^{^}

    I’m assuming the main driver is tradition, rather than an incentive to harm (e.g. due to escalating conflict, sadism, retributivism). If it was the latter, (D) would dominate (C).

^{^}

 Magnus Vinding (2020, chapter 1.2) and Willliam MacAskill (2022, p.216) have both made the case for assuming such an empirical asymmetry. Here’s an excerpt from Vinding:  “At the level of potential states, we can observe that, as philosopher Jamie Mayerfeld notes, it is a “sense shared by many people that the intensity of happiness can expand very little in comparison with the intensity of suffering.” In other words, the range of possible positive experiences does not seem to reach remotely as high as the range of possible negative experiences reaches deep. [...] Psychological evidence supports the existence of such an asymmetry at the level of potential states [...].”

^{^}

 While utilitarianism values both the reduction of disvalue and the creation of value, many longtermists value one more than the other (the former, namely; I’ve very rarely encountered people explicitly and consciously endorsing that suffering matters less than the creation of value). Magnus Vinding (2020, Part I) condenses a wild range of normative arguments in favor of prioritizing the reduction of (extreme) suffering. Regardless of people’s “favorite” moral theory, we should stay epistemically humble and moral uncertainty should be factored in, here. For what it’s worth, MacAskill et al. (2020, p.185) writes: “Under moral uncertainty you should treat alleviating suffering as more morally important than increasing happiness. [...] According to some plausible moral views, the alleviation of suffering is more important, morally, than the promotion of happiness. According to other plausible moral views (such as classical utilitarianism), the alleviation of suffering is equally as important, morally, as the promotion of happiness. But there is no reasonable moral view on which the alleviation of suffering is less important than the promotion of happiness. So, under moral uncertainty, it’s appropriate to prefer to alleviate suffering rather than to promote happiness more often than the utilitarian would.”

^{^}

 However, since technological progress will also drastically increase the impact of (A), this shift will probably not be massive.

^{^}

 However, this shift is probably not massive since (i) human values might be locked-in early on (such that it’d crystalize the influence of the preferences of (some) current humans that are quite far from being impartial and utilitarian-ish), (ii) if impartial utilitarian values are not locked-in soon, value drift seems hardly evitable and would most likely lead – all else equal – to an increased influence of (C) relative to (B) and (D), and (iii) the increasingly pronounced political polarization might maintain the force of (C) and (D).

^{^}

 This assumes that goal preservation and long-term-trajectory changes that are predictably and robustly good are tractable (and will actually be made, i.e., not all the longtermists resources will be spent on reducing risks of human extinction/collapse), which also seems possible although not obvious (see Greaves 2016). Also, in a comment on Christiano’s (2013) post, Jan raises the following interesting consideration: “Will your argument break down if immortality is invented? In this case, evolution would still select for patience, but selfish actors might have strong preferences about the long-term outcome and be patient.”

^{^}

 As implied, this argument is relevant to you only if we assume that you are committed to accepting this potential moral truth, even if it greatly differs from what your values say. If you are not committed to accepting such a moral truth, you should not account for this factor in the model, since a good future according to such a moral truth may very well be neutral or bad according to your current preferences not informed by the moral truth.

I’d also like to note that the argument relies on the following chain of facts and on the assumption that all those facts might be true:

There is a moral truth. (This has been argued by, e.g., Sharon Hewitt Rawlette (2016) as well as Magnus Vinding (2014).)

It is possible to “find it” and recognize it as such.

Future humans will actually find it.

Future humans will (unconditionally) accept it. (As an anti-example of this, Brian Tomasik (2014) writes: “Personally, I don't much care what the moral truth is even if it exists. If the moral truth were published in a book, I'd read the book out of interest, but I wouldn't feel obligated to follow its commands. I would instead continue to do what I am most emotionally moved to do.”)

Future humans will know how to “implement” this moral truth effectively.

Show all footnotes