(Cross-posted from Hands and Cities)
Life in the future could be profoundly good. Many people accept something like this in principle. But I think it often goes underestimated in practice, especially once we imagine society’s most glaring problems fixed, and ask where we might go from there. The difference in quality of life between a fixed-up version of our current world and the best possible future is, I think, less like the difference between a mediocre job and a beach vacation, and more like the difference between being asleep and being awake; between blindness and seeing; a droplet and an ocean; a cave and an open sky.
Following Bostrom (2008), let’s call a profoundly good future “Utopia.” The most important fact about Utopia, I think, is that we (that is, humanity and our descendants) could, with enough patience and wisdom, actually build it. This is a real thing we could actually do; a way the future could really be. It’s a thing the way laptops were a thing before they were invented; the way you could, if you really wanted, find a way to get some ice cream by the end of next week. It’s not some empty fantasy or symbol. It’s a physically-available path that, modulo certain exotic scenarios, is actually open to us, if we don’t mess up.
This post describes some different ways of thinking about Utopia, and what I think matters most about the differences. It also discusses some common objections to different types of “Utopian” thought and practice. My focus is on the quality of life in Utopia; see Ord (2020), Chapter 8, for more on just how much of that life there could be.
I. Concrete utopias
We can think of visions of Utopia in two broad categories: concrete, and sublime.
Concrete Utopias are imagined in specific and often human-scale terms. Most literary depictions of Utopia fall into this category, as does the vision offered by a friend of mine who hopes, if he makes it to the Utopian future, to “sit on the giant pile of pizza and play video games all day” (I think this is at least partly tongue in cheek, but it’s a bit hard to tell).
One issue people raise about these Utopias — and which the pizza example, for me, illustrates — is that they don’t always sound particularly appealing, especially not to everyone, or after scrutiny. Orwell, for example, complains that “all efforts to describe permanent happiness, on the other hand, have been failures.” See Yudkowsky, here, for discussion of some of the complexities involved.
My own worry about concrete Utopias is not that they are unappealing. Indeed, Le Guin’s Omelas, absent the child, doesn’t sound so bad to me at a glance. And regardless, whether we can describe or imagine a perfect society, from our current standpoint, seems to me like very little evidence about the degree of perfection available in principle.
Rather, my worry is that concrete Utopias are unrealistically small, familiar, comprehensible. Glancing at various Wikipedias, my sense is that literary depictions of Utopia often involve humans in some slightly-altered political and material arrangement: maybe holding property in common, maybe with especially liberated sexual practices, etc. And when we imagine our own personal Utopias, it can be easy to imagine something like our current lives, but with none of the problems, more of the best bits, a general overlay of happiness and good-will, and some favored aesthetic — technological shiny-ness, pastoralness, punk rock, etc — in the background.
This seems fine to me as a way of connecting emotionally with the possibility of a better world, or for thinking through different political possibilities. But if we start to slip into thinking that Utopia would actually be like that — e.g., that the best futures would be cleaned-up and somewhat improved versions of the world we know — then I start to worry about drastically underestimating how good — and how different — the future could be. Here a quote from Bostrom that I often think of in this context:
“What I want to avoid is to think from our parochial 2015 view—from my own limited life experience, my own limited brain—and super-confidentially [sic] postulate what is the best form for civilization a billion years from now, when you could have brains the size of planets and billion-year life spans. It seems unlikely that we will figure out some detailed blueprint for utopia. What if the great apes had asked whether they should evolve into Homo sapiens—pros and cons—and they had listed, on the pro side, ‘Oh, we could have a lot of bananas if we became human’? Well, we can have unlimited bananas now, but there is more to the human condition than that.”
Basically, many concrete Utopias sound to me a lot like the banana thing. It’s not just that they’re false in the way that all concrete visions of the future are false, or that they’re overconfident. It’s that they’re on the wrong scale, in the wrong register of comprehension. The apes who expect bananas aren’t just wrong about human food selection. They’re wrong, in a deeper way — a more exciting and important way — about what they’re getting themselves in for.
A lot of this is about the possibility of dramatic improvements to our capacities. These are notably absent from many concrete Utopias, and understandably so, given the difficulty of imagining them from our current perspective. The phrase “brains the size of planets” sounds silly, but I actually think it points in the right direction here, with the right of connotation of alien and overwhelming differences in scale and situation. Obviously, we should approach any fundamental changes to our condition with extreme caution. But the case for – ethically, responsibly, safely — enhancing our capacities, once doing so becomes possible, seems to me extremely strong, and we know of few deep limits to the project, save those we impose on ourselves, or on each other.
Some will argue that a Utopia that involves such enhancements is not worthy of the name. I disagree with this, but I won’t get into the issue now. If we constrain our scenarios to ones in which the basic biological and cognitive situation that we face remains unaltered, then concrete visions of Utopia, based on what we can currently imagine, will have a better shot at capturing what’s available. But views, like my own, that countenance the possibility (given the right types of caution, ethics, safety, etc) of significant enhancements in human capacities should, I think, be very wary of the banana thing.
I think part of what’s underlying this thought is a broader aesthetic, which I’ll call the “vastness of mind-space aesthetic,” and which also, I think, underlies a lot of worries about artificial intelligence. This aesthetic applies the kind of “holy shit things can get extremely big” lesson we’re familiar with from cosmology to the space of possible minds, in an effort to overcome similar types of parochialism (see Ord (2020), Chapter 8, for some discussion). Hence, one thinks of sufficiently advanced AI vs. human not as John Von Neumann vs. Homer Simpson, but as human vs. ant — and this only as a grossly inadequate gesture. The same expansion of scale applies to the quality of life that mind-space makes available, if we’re able to explore it.
II. Sublime utopias
Enter sublime Utopias. These visions focus less on concrete details, and more on the sense in which Utopia is incomprehensible from our current perspective. Hence the word “sublime,” which I hope will connote not just “awesomeness” but something like “beyondness” — e.g., a mental representation that foregrounds its own inadequacy, and points past itself to something overpowering and still-not-understood.
Sublime Utopias are less at risk of banana-type mistakes. Their central problem, though, is that they can lose all substance and emotional appeal altogether. Thus, if “heaven is a place where angels sing hymns, and you get to be with your family again” is a concrete vision of heaven, “heaven is eternal union with God in perfect love” is a sublime one, especially if we add “which you cannot hope to comprehend with a finite mind, but trust me it’s great.” The latter is less parochial, and less at risk of objections like “but I don’t like hymns” (though here I tend to think: man, the hymns in heaven could be so good); but it’s also less rooted in what actual humans care about, and correspondingly unexciting (the parallel holds for Hell: “separation from God” is a lot less scary than literal torture). Incomprehensibility drains content. In the extreme, sublime Utopias are pure light, with nothing to see.
Nick Bostrom’s “Letter from Utopia” is the best depiction of a sublime Utopia that I’ve encountered. I recommend the published 2008 version, still available on the Future of Humanity Institute’s website, rather than the updated 2020 version, in which some my favorite parts have been removed or edited. In particular, Bostrom provides a great example of what I see as the best method of characterizing sublime Utopias without losing all content. I’ll call this method “extrapolating the trajectory of life’s best moments.”
Peak experiences are, I think, the most straightforward input to this method, but it can also, I think, be applied to changes in the quality of e.g. our relationship, communities, and epistemologies. Running with peak experiences as our example, though, and following Bostrom: consider what happens to your mind in the moments that have most shown you what life can be at its best — moments of joy, love, beauty, energy, immensity. Moments that have made you sit up straight, and say: “Oh. Whoa. This is crazy. This isn’t some pale imitation of life; this is the real thing. This is important. I didn’t realize this was possible.” For me, such moments often have a quality of greater awakeness, aliveness, here-ness. The world is vast, and shining with beauty and meaning; and it seems, not a new world, but the world as it always has been; the world I’ve always been living in, but haven’t truly seen.
The moments most salient to others may have a different flavor. My aim is not to isolate some specific dimension along which all such moments vary, but rather to point to the familiar experience of realizing that some dimension or other that matters to the quality of our lives can vary, in the good direction, much more than you had previously encountered; that there is more to the story than you had thought — something, perhaps, that lots of people have already been talking about; something they may even be fighting for, changing their lives for; but which you hadn’t, till now, really understood.
As Bostrom notes, a central issue with such moments is that it’s hard to remember what they’re really like, once they fade. And they fade all too quickly. Life descends into something more dead, numb, thin, mundane. Bostrom calls this “soot”; Tim Urban, “fog”; someone else I know, “Dilbert dust.” For our purposes, though, the important thing about such experiences isn’t how long they last, or whether it’s worth, in our present situation, trying to make them last longer; the point is the evidence they provide about what’s possible. From Bostrom (in the 2008 version):
“Quick, stop that door! Look again at your yellowing photos, search for a clue. Do you not see it? Do you not feel it, the touch of the possible? You have witnessed the potential for a higher life: you hold the fading proof in your hands. Don’t throw it away. In the attic of your mind, reserve a drawer for the notion of a higher state of being, and in the furnace of your heart keep at least one aspiring ember alive.”
I think the notion of “proof” here is a good one. As far as I can tell, we basically have proof that life can be at least as good as the best experiences any human has ever had — and these experiences are much better, richer, vaster, more conscious and alive and awake, than can be easily appreciated when they’re not being had.
More importantly, though, I think we basically know (even if we don’t strictly have proof) that these experiences are nowhere near the limit of what’s possible; that our actual, physical universe is one that makes possible structures of consciousness and wisdom and joy and love that are vastly farther in whatever direction our minds travel, when they become more conscious, wise, joyful, loving, then anything anyone has seen so far. There are oceans we have barely dipped a toe into; there are drums and symphonies we can barely hear; there are suns whose heat we can barely feel on our skin.
From William James:
“Our normal waking consciousness… is but one special type of consciousness, whilst all about it, parted from it by the filmiest of screens, there lie potential forms of consciousness entirely different.”
Indeed, some possible forms of consciousness may be simultaneously extremely good, and utterly unlike anything any creature on earth has ever touched. Oceans, not that we haven’t waded into, but that we don’t even know exist.
That said, I’m inclined to think that Utopia, however weird, would also be, in a certain sense, recognizable — that if we really understood and experienced it, we would see in it the same thing that made us sit bolt upright, long ago, when we first touched love, joy, beauty; that we would feel, in front of the bonfire, the heat of the ember from which it was lit. There would be, I think, a kind of remembering. As Lewis puts it: “The gods are strange to mortal eyes, and yet they are not strange.” Utopia would be weird and alien and incomprehensible, yes; but it would still, I think, be our Utopia; still the Utopia that gives the fullest available expression to what we would actually seek, if we really understood.
Perhaps there are arguments out there that this sort of “sublime Utopia” oversells what’s at stake (Internet, I’m curious to hear what you think are the best ones). My current best guess failure mode would be one in which it turns out that humans, for some reason, don’t or can’t value things very different in depth and intensity from what we have now. That is, my best guess is that if what I’ve said about un-swum oceans and unheard symphonies turns out false, it’s false for ethical rather than metaphysical or physical reasons; false because our values are small, not because the space of possible modes of consciousness is any less large. The human brain, after all, is an extremely specific and limited organ: it has some 100 billion neurons, it uses some 20 Watts of power, its signals travel, at best, at some hundreds of meters per second. These are nowhere near physical limits. If small changes in chemistry can produce the dramatic variation we actually witness in quality of human mental life; if small (relative to what’s possible) changes in brain architecture and cell count can move us from ants to mice to monkeys to humans; then think of what a sustained, serious (cautious, ethical, responsible) effort to explore the full range of what’s possible is likely to reveal.
III. Rapture of the nerds?
As far as I can tell, this is a real thing. It’s not some woo thing that the hard-nosed people understand to be silly. There’s no fancy metaphysics here. We don’t need to think that peak experiences reveal any special insight or knowledge unavailable to science or the mundane world. We don’t need to think that any technologies are around the corner. We don’t even need to be realists about consciousness. All we need to think is that whatever happens in our brains and bodies, during the events humans label “peak experiences,” is movement along some trajectory we care about; and that it is possible for bodies, brains, and other relevantly similar physical structures to travel much, much further in that direction. This seems to me like the actual situation we’re in.
All the same, people object to visions of Utopia — and especially, sublime ones — on the grounds of their spiritual connotations. “Rapture of the nerds,” people say (though this phrase can be applied to many things). Or, maybe more specifically: it all sounds a bit too much like the thing people hoped for out of Heaven, or that people get all excited about when they’re in some kind of “spiritual” mode. Better to keep your feet on earth.
It is certainly possible relate to the possibility of Utopia using archetypes, impulses, and forms of psychological orientation familiar from religious and spiritual contexts. As a small example: as with Heaven, it can be easy, in the face of Utopia’s incomprehensibility, to round it off, in the imagination, to something like an abstract wash of perfection or oceanic light, maybe with a few incomplete images thrown in; or to think of it as a place where all problems are solved, all desires fulfilled. But unlike Heaven, Utopia, if something like it ends up getting built, will be a specific, concrete, physical world, with attendant frictions and problems, idiosyncrasies and contingencies; its own ways of distributing resources, resolving/preventing conflicts, and so on; and ultimately, with fundamental limitations on what can be done. There are no promises of perfection here, or of eternity.
More broadly, some possible failure modes in relating to Utopia do seem similar to failure modes associated with religion (though some of these are also failure modes of believing, more generally, that something is extremely important). I discuss a few of these in the next section.
At a higher level, though, I want to note that I actually think it fine and appropriate to direct certain attitudes familiar from religious and spiritual contexts towards something like Utopia. In particular, I think Utopia an appropriate object of hope, wonder, humility, and purpose. It is, in the words of Carl Sagan, a “worthy goal.” What’s more, while the forms of consciousness, love, joy, and wisdom available in Utopia are not transcendent of the natural world, they are transcendent, in some basic sense, relative to us. To point at them, we need to point past ourselves, along the line we’ve already seen ourselves travel, but beyond our own current limits, to something rightly, I think, thought of as higher, deeper, more awake, more real. Spiritual and religious contexts are the primary place humans have gone to develop tools and practices for doing this type of thing.
Indeed, at their best (in my view), religion and spirituality express an aspiration to look ultimate reality in the eye; to live in the light of our actual existential condition, seen, as much as possible, in its entire. Utopia, I think, is the place for this too — and in that sense, the proper object of something closely akin to religious aspiration. Now we see in a mirror dimly. Then, face to face. (Or at least, much less dimly.)
Some will eschew attitudes that smack of religion and spirituality regardless. That’s OK too. The main thing is not shrink your sense of what’s possible in doing so; not to put the wrong limits on what can be really hoped for, what can actually be a thing in this plain old atoms-and-stuff itchy-feet Donald Trump gotta-wash-the-dishes universe, because it sounds too far outside the bounds of the mundane, the familiar — and hence, one might subtly assume, the secular.
IV. Does this matter? Is it good to think about?
Do visions of Utopia matter? To work towards a better world, after all, we don’t need to ask, in any detail, just how much better it could be — what our efforts could ultimately lead to. We just need to keep moving in the right direction. Perhaps, then, this kind of thinking is a distraction or worse, especially given how pressing our actual, present problems are; how much we seem at risk of futures much worse than Utopia; how idle such speculations can seem, as the world burns.
Indeed, the project of trying to sketch a plan for something like Utopia, and to build it now, here on earth, has a very bad track record. “Utopian” thinking has an aura, at best, of a kind of thin-ness and naïveté — an aspiration to encompass all of the world’s mess and contradiction in a top-down, rational plan, to “solve things once and for all,” to “do it right this time.” Here one thinks of poorly but “rationally” planned cities, failed communes, and the like.
And at worst, of course, Utopianism ends in abject horror. There is a certain type of fervor that comes from thinking, not just that paradise is possible, but that you and your movement are actually, finally, building it. In the grip of such enthusiasm, humans have historically been very willing to do things like “purge” or “purify”; to crush and suppress any opposition; to break eggs to make omelets, and so on. As Kundera writes:
“Totalitarianism is not only hell, but also the dream of paradise — the age old dream of a world where everybody would live in harmony, united by a single common will and faith, without secrets from one another. . . If totalitarianism did not exploit these archetypes, which are deep inside us all and rooted deep in all religions, it could never attract so many people, especially during the early phases of its existence. Once the dream of paradise starts to turn into reality, however, here and there people begin to crop up who stand in its way, and so the rulers of paradise must build a little gulag on the side of Eden…”
We’ve seen what humans marching under Utopian banners have wrought; our historical immune system has left us wary.
And there are subtler objections to dreams of Utopia as well. Paying lots of attention to how much better life could be can smack of greed and dissatisfaction; of wanting rather than appreciating; of ambition and striving rather than humility and contentment. Le Guin, in characterizing what Utopia has been (though need not be), adds a slew of further adjectives: “Euclidean,” “masculine,” “European,” “dry,” “aggressive,” “lineal,” “expanding,” “advancing.” One wants, perhaps, more earth, dirt, ground. The place to live and love and work is here; and here, one might think, risks neglect or worse, if we set our gaze on some imagined and unknowable future.
There’s lots to say about these sorts of objections. For now, the main thing I want to emphasize is that I don’t think we should spend much time now trying to figure out in any detail what Utopia looks like; that task is for a species much wiser and safer than we are at present. The main thing for us to do about Utopia, I think, is to protect its potential to someday be made real: and that does, indeed, mean living and loving and working here, in the present.
But keeping in mind a basic picture of what’s at stake is important. This is partly because it matters to the type of hope one has; the type of story one thinks of human pain and joy as ultimately a part of; the type of victory one thinks it possible, amidst all the world’s darkness, to fight for, and to win. But more broadly, I think, the possibility of a profoundly, unimaginably good future is, modulo certain exotic scenarios, just a basic fact, a feature of our actual situation that we need to act in light of. For the normal, mundane reasons that apply to most basic but possibly decision-relevant facts, then, I think it’s worth being clear about.
Great piece. I really connected with the part about the vastness of the possibility of conscious experience.
It sounds a little bit like you're saying that utopia would be recognisable to modern day humans. If you are saying that, i'm not sure I would agree. Can a great ape have a revelatory experience that a human can have when taking in a piece of art? There exists art that can create the relevant experience, but I highly doubt if you showed every piece of art to any great ape that it would have such an experience. Therefore how can we expect the experiences available in utopia to be recognisable to a modern day human?
This post is beautiful, rational, and useful - thank you!
As the beginning of a reply to the question "What does a “realistic best case transition to transformative AI” look like?", we could maybe say that a worthwhile intermediary goal is getting to a Long Reflection when we can use safe (probably narrow) AIs to help us build a Utopia for the many years to come.
i believe, just like much aspects of the future, utopia is hard to envision. as humanity progresses over time, we will eventually either die out or reach a state of utopia (the latter is way more likely considering we will probably not go extinct in the next 5,000 years)
how such a utopia would be like is unknown, but i know it would exist and probably on the basis the same trends that are happenning over time
predicting the future is hard in general though
Nice post - I think it is good to reflect on what we might achieve, rather than only focusing on reducing X risk.
I’m not sure how feasible these are, but would personal universes address some of these issues?
Thanks :). I haven't thought much about personal universes, but glancing at the paper, I'd expect resource-distribution, for example, to remain an issue.
Thanks! I relate to the last paragraph, it felt motivating and important to think about a positive vision that we’re working for. I think your essay is a really nice introduction for that.
As a side note, I really like Scott Alexander‘s Archipelago of Civilized Communities, an Utopian-ish vision where communities can freely form on their own islands but people are free to always leave. It’s probably only one level above the „my house is made of banana pizza“ brain expansion level of monkey Utopias, but it could be a great first step! https://slatestarcodex.com/2014/06/07/archipelago-and-atomic-communitarianism/