This is a crosspost from my blog post.

In this post, I will summarize William MacAskill’s “Better Futures” essay series, which is one of the most important works in field of longtermism because it substantially expands upon intellectual thought related to how much value we should expect the far future to have and the extent to which our actions today can affect that value.

For a broad overview:

1. The first essay “Introducing Better Futures” argues that working to improve futures where humanity survives could have a similar value to ensuring humanity’s survival.
2. The second essay “No Easy Eutopia” argues that the best possible future is likely a very narrow target that humanity would have to intentionally aim at.
3. The third essay “Convergence and Compromise” argues that it is very possible that humanity will fail to reach such a narrow target.
4. The fourth essay “Persistent Path-Dependence” argues that there will be actions available to humans this century that will enable them to predictably influence the expected value of the far future.
5. The fifth essay “How to Make The Future Better” lists actions that can be done today, which might predictably improve the far future.
6. The last essay “Supplement: The Basic Case for Better Futures” expands upon arguments in the first essay.

(As an aside, this essay series had Fin Moorhouse as a co-author for two essays and Philip Trammel as a co-author for one essay. In this post, I will only summarize the first five essays, since the final essay mostly recaps ideas from the rest of the series.)

Introducing Better Futures

We can think of there as being roughly two ways to improve humanity’s far future

First, you can work on humanity’s survival, which is to say, reducing the risk that humanity permanently enters a near-zero value future. (This could, for example, include reducing the risk of human extinction or the permanent collapse of civilization.)

Second, you can work on humanity’s flourishing, which is to say, increasing the value of humanity’s future over the long-term given that it survives. (This could, for example, include working to ensure that future humans have the correct moral views related to digital beings.)

In MacAskill’s view, we should consider work on survival and flourishing to be of similar value on the basis of the scale, tractability, and neglectedness framework, which is a framework for deciding which cause areas to work on.

Scale

If you think that, by default, humanity has an 80% chance of survival but that it will, in expectation, only achieve 10% as much value as it can if it survives, then you should consider flourishing to have far greater scale than survival.

This is because, if you reduce the risk of extinction by one percent, that will have far less value than increasing the value of humanity’s future by one percent.

As MacAskill writes, “On these numbers, if we completely solved the problem of not-Surviving, we would be 20 percentage points more likely to get a future that’s 10% as good as it could be. Multiplying these together, the difference we’d make amounts to 2% of the value of the best feasible future.

In contrast, if we completely solved the problem of non-Flourishing, then we’d have an 80% chance of getting to a 100%-valuable future. The difference we’d make amounts to 72% of the value of the best feasible future — 36 times greater than if we’d solved the problem of not-Surviving. Indeed, increasing the value of the future given survival from 10% to just 12.5% would be as good as wholly eliminating the chance that we don’t survive.3”

Neglectedness

Work on flourishing is much more neglected than work on survival. For instance, longtermists seem to overwhelmingly focus on extinction risks.

Tractability

MacAskill argues that it is far less clear how tractable Flourishing is and that this is “the strongest argument against the better futures perspective”

MacAskill concludes by stating that he believes we should not focus on achieving a narrow view of what the best possible future looks like but that, instead, we should work towards creating viatopia, a temporary state of society where it can guide itself towards a near best future.

No Easy Eutopia

According to the Better Futures perspective, flourishing is only important if the future has an expected value significantly lower than its maximum possible value.

This could be the case if:

1. The best possible future is an extremely narrow target.
2. Humanity does not determine what this target is, cannot work towards this target, or lacks motivation to work towards this target.

In this essay, MacAskill and Moorhouse (who I will refer to as M&M) make the case that the first claim is likely true, and they give five reasons to think this is the case.

First, there is a high risk that the future could have far less value than it otherwise would have if it had an on-going moral catastrophe. Throughout history, humans have engaged in immoral practices such as discrimination, genocide, and slavery, which made their societies have far less value than they otherwise would have. This is also true during the modern day according to most moral views. For instance, if you believe that abortion constitutes murder, then the “over 70 million (annual) abortions” would constitute a moral catastrophe. If the future were to possess such a moral catastrophe, it would lost out on a vast amount of value.

Second, the traditional view of utopia misses out on most potential value. For instance, if we made Earth a living paradise but did not settle the rest of the affectable universe, most potential value could be lost.

Third, there are a wide range of possible ways the future could have a moral catastrophe. For instance, we could have the incorrect view on population ethics and create the incorrect number of people relative to the number of resources that we have. We could also have the wrong views on how to treat digital beings or what the right form of wellbeing is.

Fourth, the value of the future could be a product of many different factors. Even if all factors are close to as good as they could be, their product could still be shockingly low. (For instance, (.9)^(10) = .35) One reason to think this could be the case is that, for modern humans, even if someone has many positive aspects to their life, a single negative aspect, such as chronic pain, significantly reduces their wellbeing.

Lastly, it could be that, from a psychological perspective, we are inclined to think that a near best future is easy to achieve but that it actually is not because, if we experienced such a future, we would determine that it could actually be much better, similar to a hedonic treadmill.

Convergence and Compromise

In this essay, M&M argue that it is unlikely that humanity will get a near-best future by default since not enough people will converge to correct moral views or be motivated to act on them.

More precisely, they argue that humanity could have a mostly-great future if:

1. The world has WAM-convergence, which is to say that there is widespread convergence on accurate moral views, and people would be motivated to act upon them.
2. There is partial AM-convergence combined with moral trade, which is to say that some people come to the correct moral views or are motivated to act upon them and that they make mutually beneficial moral trades.
3. No one comes to the correct moral view or is motivated to act upon it, but we still get a mostly-great future anyways.

They argue that there are possible blockers to each of these scenarios, and they regard the final scenario as highly unlikely.

Will WAM-convergence occur?

M&M assess five reasons we might see WAM-convergence. They argue against these and then offer two more reasons to think that we will not get WAM convergence. (For context, although they are quite critical of WAM-convergence, they are not rejecting it entirely.)

Current agreement

One may think that WAM-convergence will likely occur because most people today agree on a wide range of moral topics. As such, it seems likely that humanity will continue to progress towards WAM-convergence. M&M reject this view on the basis of four reasons.

First, although there is significant moral agreement today, it has been on instrumentally valuable rather than intrinsically valuable goods. In the future, when people are able to optimize towards intrinsically valuable goods rather than instrumentally valuable ones, we will have far less convergence than we do today. (For instance, if you and I both agree that helping people is important, but I only think it’s important because I want to make people happy, and you believe it’s only important because we have a moral obligation to help others, in the future, our views will diverge as I am able to optimize towards producing happiness without helping others.)

Second, there are currently pressures that cause people to have very similar moral beliefs. For instance, most people don’t believe “highly unusual moral views” because they risk being seen negatively by others. If these pressures go away, people’s moral views could significantly diverge.

Third, it could be that the current level of agreement is only due to a lack of moral reflection. It could be that, if people have more time to reflect on their views, they may be more likely to develop views that are widely different from each other.

Lastly, even if the current agreement indicates that there will be more agreement, it doesn’t necessarily mean that people will have complete agreement, which is necessary for a near best future. Even if I agree on 99% of things with you, your best possible futures may have roughly no value in my eyes and vice versa.

Moral progress

Another second one may think that WAM convergence will occur is that there appears to have been significant moral progress over the last couple hundred years. M&M offer three reasons against this view.

First, “even if world values seem to be converging, that doesn’t mean they’re converging towards the right views.” For instance, it could be that modern views are a result of contingent historical events rather than a steady march of progress. They write, “if Germany had won WWII, or the USSR had won the Cold War, then modern values might be much more authoritarian.”

Second, it could be that we’ve actually had minimal moral progress. For instance, according to total utilitarianism, it is a moral catastrophe that fertility rates have been dropping because fewer humans have been born to live happy lives.

Third, it could be that there is a mechanism that drives moral progress, but this mechanism may not necessarily cause complete moral progress. For instance, the use of protest and persuasion have allowed minority groups to reduce their own oppression. This option is not available to animals, and may not be available to digital people so they may be unable to make use of this mechanism.

ASI-assisted reflection

A third reason one may expect WAM-convergence is that ASI-assisted reflection may enable humans to come to correct moral views since ASI could help people to avoid errors in reasoning and engage in extensive reflection for the use. If people engaged with these machines and wanted to learn what the correct moral view is, they could enable them to do so.

MacAskill and Moorhouse offer a few reasons to think this is not the case:

1. ASI-assisted reflection may not result in moral convergence because people may “have different ethical starting points and endorse different reflective processes,” which would lead them to different conclusions.
2. “People might just be uninterested in acting on the results of an open-ended process of reflecting on the good “
3. “People might just not be interested in acting in accordance with the result of any kind of ethical reflective process.” For instance, if you knew that a moral reflection would lead you to take a different action than you currently desire to take, would you engage in such reflection?
4. People might only use reflection as a means to satisfy their own self-interested preferences.
5. People might not engage in such reflection because, after all, modern people also don’t really care about figuring out what’s morally true.

Abundance and diminishing returns

A fourth reason one may expect WAM-convergence is that, if people have both self-interested and altruistic preferences but their self-interested preferences experience slower diminishing marginal utility than their altruistic preferences, then, over enough time, virtually all of their resources would go towards altruistic preferences.

To counter this idea, M&M point out that:

1. This argument only really supports the view that preferences with slower diminishing marginal return will get the most resources put towards them.
2. Just because the preferences are altruistic doesn’t mean that they are morally correct.

Long views wins

A fifth reason to expect WAM-convergence is that, since altruistic views favor saving resources for the future and views that favor saving resources for the future tend to win out over long enough time periods, we should expect altruistic views to win out.

To counter this argument, M&M point out that this only really implies that views with low-discount rates will win out, which may not be necessarily altruistic views. Additionally, even though these views may win out over long enough time periods, they may not have enough time if lock-in events occur soon.

An argument against WAM-convergence

M&M then argue that we shouldn’t expect WAM-convergence given moral realism or moral anti-realism.

If moral realism is true, they argue that the correct moral view will strike people as really strange so they won’t be motivated to act on it, and, if this belief is motivating, they might even seek to not know what it is.

If moral anti-realism is true, since there is no one correct belief, we probably shouldn’t expect people to converge to any specific view.

Blockers

In this section, M&M argue that, even if we get WAM-convergence, we might not get a near best future. This could occur if humanity is unable to “choose its future” due to “evolutionary forces.” This could also occur if people have incorrect “non-moral views,” such as incorrect views about the nature of reality. Lastly, this could occur if the world becomes locked-in before enough time has passed for WAM-convergence to occur.

What if some people aim at the good?

In this section, M&M argue that partial AM-convergence (which happens if some people accurately converge to the correct moral view, are motivated to act upon it, and use moral trade) may result in a “mostly-great future.”

If people in the future have different moral values, they could engage in moral trade, which means doing something good in the view of another so that they do something that is good according to you.

Unfortunately, at the same time, if this were the case, they may also be able to engage in moral threats, which means threatening to do something bad in the view of another so that they comply with your demands. M&M consider threats to be a major issue.

According to M&M, we should work to ensure moral trade is possible while also preventing moral threats.

They also point out that moral trade could face certain blockers, such as (besides the ones in the previous section), concentration of power and poor collective decision making procedures. If too few views are represented by those in power, no one with the correct moral view may be able to engage in moral trade. Similarly, if we have poor collective decision making procedures, those with the correct moral views may be unable to properly influence decision making.

What if no one aims at the good?

M&M argue that we are unlikely to get a mostly great future if no one aims at the good, since, according to the previous essay, we should expect the near best future to be a narrow target. They point out that people’s self-interest or use of instrumental goods that are actually intrinsically good could make the future better, but they argue that this is unlikely to get us to a mostly great future.

Conclusion

They conclude by suggesting that, given these arguments, we should think that we are unlikely to get a near-best future, which helps the argument that we should work to improve the far future over working on survival since we’re far closer to the ceiling on survival than on flourishing.

Persistent Path-Dependence

In this essay, MacAskill argues that, this century, there is a significant likelihood that events will be occur that persistent path dependence (PPD), which means that they will cause the world to enter a path it otherwise may not have, that their effects that will be extremely persistent, and that they will “predictably influence the expected value of the future.” (For context, I don’t think that he makes any arguments supporting the third claim.)

Human extinction and AI takeover

MacAskill begins by doing a detour and assessing the value of futures if humanity does not survive.

He argues that, if humans go extinct, intelligent life may still re-evolve on Earth or aliens may eventually take over our part of the cosmos.

He also points out that, if AI takes over, it may still create a future civilization.

Based on how good or bad you think these scenarios are, they might change how bad you think extinction or AI takeover really are.

Mechanisms for persistent path-dependence

MacAskill then resumes his central argument. In this section, he argues that, this century, some people will be able to take actions with PPD because they will gain a greater ability to control the future and to prevent disruption.

Greater control over the future

Currently lack significant “power to control the future” because they don’t have the technology to do so. For instance, some people would like to permanently entrench their values but they are unable to do so because they eventually will die and the next generation will distort their values. In this section, MacAskill offers some ways that, this century, people could overcome these obstacles.

First, people could create AGI-based institutions that permanently lock-in their values by having them programmed into AGI systems that are unable to be overthrown. He argues this could occur as the result of a global autocracy, a country doing this now and then eventually becoming dominant or “able to stably retain a share of global power,” or from countries using “indefinitely-binding AGI-enforced treaties.”

Second, People could also permanently lock-in their values by designing digital beings whose “values and decision making processes (could) … persist unchanged for potentially millions of years.” It could also occur as the result of “biological immortality.”

Third, people could design beings to have a specific set of values. This could cause PPD if a dictatorship designs its citizens or if people design future generations.

Lastly, strong self-modification could occur in the case of digital or biological beings. People might alter their values or preferences so that they are unwaveringly committed to a certain belief. Furthermore, “Even if people don’t lock in to particular beliefs or preferences, there could still be strong path-dependency of their final beliefs and preferences based on their initial beliefs and preferences” if people’s changes cause further changes to be more or less likely.

Less disruption

Rulers are also unable to control the future because of external disruptions from issues such as environmental changes, rebellions, technological changes, and interference from outside countries. In this section, MacAskill offers some ways that technological changes could reduce these disruptions.

First, extreme technological advancement could reduce disruption since, if the rate of technological development reduces from reaching technological maturity or diminishing returns from research, there could be far less disruption from technology. Additionally, AI could enable rules to make highly accurate predictions, which would enable them to sustain their power.

Second, global concentration of power could reduce disruption since competition between different actors is a major cause of change.

Lastly, defense dominance could reduce disruption, since, if a group of people are able to defend their resources more effectively than attacking others, they may be able to stably maintain their share of resources without disruption. This could be the case if “widespread space settlement occurs” and it turns out that star systems are defense dominant.

Persistent path-dependence is likely soon

MacAskill argues that PPD is likely soon since, if AGI is developed, it could cause an intelligence explosion that could lead to an extreme concentration of power, immortality, and the acquisition of defense-dominant space resources.

He additionally notes that, since rulers historically and in the modern day have sought to permanently entrench their values, we should continue to expect people to do so this century.

How to Make the Future Better

In this essay, MacAskill argues that there are actions available to us that can positively shape the far future in ways other than reducing the risk of extinction. These actions he says are more like potential ideas that could be wrong rather than robustly good actions that have been given extensive consideration.

Keeping our options open

MacAskill begins by offering four ideas for actions that “[keep] our options open,” which I presume means preventing humanity from entering into states that it cannot get out of.

First, he suggests working to prevent a post-AGI autocracy. He argues we could work to do this by preventing democracies from becoming autocratic, preventing autocracies from entrenching their power with AI, and preventing autocracies from gaining absolute power post-AGI. These actions could be done by preventing AI misuse, creating tools that help people to resist authoritarian rule, ensuring democratic countries win the AGI race, and working to make sure that the power governing AI is widely distributed.

Second, he suggests working to delay the use of space resources by raising awareness about how an intelligence explosion could be imminent, “advocating for norms around small-scale uses of space resources that would scale desirably to large-scale uses of space resources,” and increasing public discussion on this issue.

Third, he suggests ensuring that countries have more “explicitly temporary commitments,” since it could be a good idea to delay permanent treaties when we currently don’t know what the right decision is.

Lastly, he suggests working to “slow the intelligence explosion,” which he argues a country could do if they have a major head start or are able to get an agreement with competitors.

Steering our trajectory

In this section, MacAskill offers six ways that we could “try to ensure that, if civilisation is pushed into one particular path, that path is better rather than worse.”

His first suggestion is to improve the governance of the development of ASI since whoever develops ASI first might become the default world government as a result of how it would give an immediate and decisive advantage over all competitors.

His second suggestion is to ensure AGI has strong value-alignment by giving it a strong moral character, having it encourage people to make just actions, and making it motivated to do good. He also points out that, if humanity goes extinct or AGI takes over, this would increase the likelihood that AGI builds a good civilization.

His third suggestion is to create more pro AI welfare policies and to encourage more discussion on AI rights. He argues this on the basis that most future beings will be digital since they can survive easily in far more environments than humans and that “it’s much easier for them to travel across interstellar distances.”

His fourth suggestion is to work towards better space governance, since how space resources are used will determine how the future will go.

His fifth suggestion is to improve collective decision making since “most major collective decision-making uses processes that do very poorly at either representing the will of the people, or enabling the best arguments to win out.”

His last suggestion is work towards preventing sub-extinction catastrophes, since if a catastrophe occurs, the world may become less democratic and develop worse values to guide the future.

Cross-cutting interventions

He then suggests two interventions that both keep our options open and steer our trajectory.

First, he supports work on the use of deliberative AI since it comes with both major risks and major benefits. Some risks he cites are the creation of super persuasion, memes which are highly virulent but false, and a world where people don’t know what’s true since AI generates highly convincing but false arguments. The benefits he cites are society being able to recognize and solve major problems, avoiding serious moral errors, and helping people to engage in moral reflection. He argues we can work on this by ensuring a faster development of deliberative AI, ensuring the widespread use of deliberative AI, ensuring widely used AI models are helpful with deliberation, and using AI to do macrostrategy research.

He also suggests empowering responsible actors since they are able to do good across many of the problems discussed in this essay as well as others.