(This post also has a Russian version, translated from the present original by K. Kirdan.) 

Summary: The Grabby Values Selection Thesis (or GST, for short) is the thesis that some values are more expansion-conducive (and therefore more adapted to space colonization races) than others such that we should – all else equal – expect such values to be more represented among the grabbiest civilizations/AGIs. In this post, I present and argue for GST, and raise some considerations regarding how strong and decisive we should expect this selection effect to be. The stronger it is, the more we should expect our successors – in worlds where the future of humanity is big – to have values more grabbing-prone than ours. The same holds for grabby aliens relative to us present humans. While these claims are trivially true, they seem to support conclusions that most longtermists have not paid attention to, such as “the most powerful civilizations don’t care about what the moral truth might be” (see my previous post), and “they don’t care (much) about suffering” (see subsequent post).

The thesis

Spreading to new territories can be motivated by very different values and seems to be a convergent instrumental goal. Whatever a given agent wants, they likely have some incentive to accumulate resources and spread to new territories in order to better achieve their goal(s).

However, not all moral preferences are equally conducive to expansion. Some of them value (intrinsically or instrumentally) colonization more than others. For instance, agents who value spreading intrinsically will likely colonize more and/or more efficiently than those who disvalue being the direct cause of something like “space pollution”, in the interstellar context. 

Therefore, there is a selection effect where the most powerful^[1] civilizations/AGIs are those who have the values that are the most prone to “grabbing”. This is the Grabby Values Selection Thesis (GST), which is the formalization and generalization of an idea that has been expressed by Robin Hanson (1998).

We can differentiate between two sub-selection effects, here:

• The intra-civ (grabby values) selection: Within a civilization, we should expect the agents who have the values that are the most adapted to survival, replication, and expansion to eventually be selected for. In the absence of early value lock-in
  , this seems to favor grabby values, since those with the grabbiest values will be the ones controlling the most resources, all else equal. Here is a specific plausible instance of that selection effect, given by Robin Hanson (1998): “Far enough away from the origin of an expanding wave of interstellar colonization, and in the absence of property rights in virgin oases, a selection effect should make leading edge colonists primarily value whatever it takes to stay at the leading edge.”^[2]
• The inter-civ (grabby values) selection: The civilizations that end up with the most grabby-prone values will get more territory than the others.

Do these two different sub-selection effects matter equally? My current impression is that this mainly depends on the likelihood of an early value lock-in – or of design escaping selection early and longlastingly, in Robin Hanson’s (2022) terminology – where “early” means “before grabby values get the time to be selected for within the civilization”. ^[3] If such an early value lock-in occurs, the inter-civ selection effect is the only one left. If it doesn’t occur, however, the importance of the intra-civ selection effect seems vastly superior to that of the inter-civ one. This is mainly explained by the fact that there is very likely much more room for selection effects within a (not-locked-in) civilization than in between different civilizations.^[4]

GST seems trivially true. It is pretty obvious that all values are not equal in terms of how much they value (intrinsically or instrumentally) space colonization, and that those who value space expansion more will expand more. This thesis is based on nothing but these very simple and uncontroversial premises. What might lead to some controversy, however, is asking How strong is the selection effect? Do value systems significantly/drastically differ in how grabbing-prone they are? The next section shallowly addresses this.

How strong is the selection effect?

The previous section already touches on the importance of the intra-civ and inter-civ selection effects relative to each other. Here, we'll consider their absolute importance, i.e., the extent to which we should expect a large number of values to actually be selected against.

I see two reasons to believe the intra-civ selection, specifically, would be more intense than what we may intuitively think:

• (i) we arguably should expect the development and deployment of something like transformative AI to make the intra-civ grabby values selection process particularly fast and strong;^[5] and
• (ii) as Hanson (2009) argues, “This is the Dream Time” for many present (and past) humans, who have the unusual luxury of being able to value and believe pretty much anything. Agents trying to survive/succeed in a civilization that is optimizing for space colonization probably won’t have this privilege, such that their values are more likely to be determined by evolutionary pressures.

One reason to suppose there wouldn’t be that much (both intra-civ and inter-civ) selection favoring grabby values is that advanced civilizations might eventually converge on a moral truth. However, the previous post within this sequence argues that this is relatively unlikely, partly because there is no reason to assume values aligned with a (potentially discoverable?) moral truth will be more competitive than those that are the most grabbing-prone.

A better reason to believe there wouldn’t be that much (intra-civ and inter-civ) selection favoring grabby values, is that agents with values that might a priori seem less grabbing-prone could still prioritize colonizing space, as a first step, to not fall behind in the race against other agents (aliens or other agents within their civilization), and actually optimize for their values later, such that there is little selection effect. Call this the convergent preemptive colonization argument. Earlier, I wrote: 

For instance, agents who value spreading intrinsically will likely colonize more and/or more efficiently than those who disvalue being the direct cause of something like “space pollution”, in the interstellar context. 

While these two specific value systems seem to differ greatly in terms of how grabbing-prone they are, the convergent preemptive colonization argument suggests that others might differ much less. For instance, Carl Shulman (2012) argues we should expect agents who want to maximize the number of “people leading rich, happy lives full of interest and reward” (“Eudaimonians”) to be nearly as grabbing-prone as agents who purely want to expand (the “Locusts”). And although I believe his argument to be far from unassailable, Shulman tells a thought-provoking story that reminds us of how much of a convergent instrumental goal space colonization still is for various value systems.

So the relevance of both the intra-civ and inter-civ selection effect might highly depend on the specific values our minds entertain while thinking about this. 

Conclusion

The Grabby Values Selection Thesis seems trivially true, but I am pretty uncertain about the significance of the selection effect. Its relevance might vary a lot depending on the exact values we are considering and “making compete” against one another. In future posts, I will investigate the significance of the selection effect, given value variations on different axes.

I warmly welcome any consideration I might have missed. More research is needed, here.

Although uncertainty is big, the more significant the selection effect, the more this has crucial implications for longtermists. My future posts will also touch on these implications and what GST tells us about the values we should expect our successors – and grabby aliens – to have. 

Acknowledgment

Thanks to Robin Hanson for our insightful conversation on this topic. Thanks to Antonin Broi, Maxime Riché, and Elias Schmied for helpful comments on earlier drafts. Most of my work on this sequence so far has been funded by Existential Risk Alliance. 

All assumptions/claims/omissions are my own. 

1. ^{^}

   By “the most powerful”, I mean “those who control the most resources such that they’re also those who achieve their goals most efficiently.”

2. ^{^}

   Other pieces have pointed at potential dynamics that are fairly similar/analogous. Nick Bostrom (2004) explores “scenarios where freewheeling evolutionary developments, while continuing to produce complex and intelligent forms of organization, lead to the gradual elimination of all forms of being that we care about.” Paul Christiano (2019) depicts a scenario where “ML training [...] gives rise to “greedy” patterns that try to expand their own influence.” Allan Dafoe (2019; 2020) coined the term “value erosion” to illustrate a dynamic where “[j]ust as a safety-performance tradeoff, in the presence of intense competition, pushes decision-makers to cut corners on safety, so can a tradeoff between any human value and competitive performance incentivize decision makers to sacrifice that value.”

3. ^{^}

   They arguably have already been somewhat selected for, via natural and cultural evolution (see Will Aldred's comment) long before space colonization becomes a possibility, though.

^{^}

Thanks to Robin Hanson for pointing out this last part to me, and for helping me realize that differentiating between the intra-civ selection and the inter-civ one was much more important than I previously thought.

^{^}

Dafoe (2019, section Frequently Asked Question) makes an analogous point.

Show all footnotes