Great article, very well said!
I would especially emphasize the human/non-human dynamics. In a sense, we are the TAI of the animal kingdom, and the results have been disastrous for every species we happened to decide to factory farm. Do unto others as you would have others do unto you comes to mind.
Also, for a more artistic and entertaining take on the veil of ignorance, check out https://www.astralcodexten.com/p/being-john-rawls
Executive summary: The author argues that the more radical change we expect from AI, the more our future uncertainty comes to resemble Rawls' "veil of ignorance," and the more we should structure society as if we might end up as any randomly selected member of it.
Key points:
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.
I loved reading this piece. It really goes to show the importance of getting a headstart when it comes to AI legislation before the veil falls and individual human instincts muddy the waters.
This is a crosspost from the Forward Pass substack with the author's permission.
What should you expect your life to look like in the future? Almost everybody is overconfident about what their life will look like in the next 30 years. For most people this is due to an underappreciation of the transformative potential of AI, but for others it’s due to an underappreciation of how unique their current positions are.
For anything, the greater change you anticipate, the more you should widen your range of possibilities to expect. When walking in a random direction, a good guess for how your surroundings will look tomorrow is what they look like today. But if you’re flying a plane in a random direction, where you are today tells you very little about what your surroundings will look like tomorrow. A better estimate would be the average of what surroundings look like everywhere on Earth. Visually, this looks like a smearing out of our expectations about the future.
Similarly, the more change we expect from AI (and we should expect a lot), the more we should expect our lives to resemble the average baseline of a life and less like our particular current one. Taken to the extreme, this would imply complete uncertainty over our future position in our society and the world.
The philosopher John Rawls argued that the most just structure for a society would be the structure chosen by people before they knew which position they’d occupy within it, which he called being behind a “veil of ignorance”. In this position, each person’s self-interest naturally aligns with what is good for everyone. The more change we expect from AI, the greater our uncertainty about where we’ll end up, and the more our actual situation comes to resemble being behind Rawls’ veil. While this uncertainty is unnerving, it also gives us a rare window to think about how to structure society with less of the bias of our current positions, at a moment when many people’s self-interest structurally aligns with the common good. We should not let this window go to waste.
People are already becoming much more uncertain about their jobs. The major AI labs all have the explicit goal of creating a machine that can do any cognitive task a human can do, but better and cheaper, and they appear likely to achieve this within the next handful of years.
It’s unclear exactly how this will affect jobs and wages, but if you make a living using your brain, you should be uncertain and concerned about whether it will continue to provide the income you’re used to.
We often assume that cognitive labor is inherently uniquely valuable, but this is just an artifact of supply and demand in modern history. Many jobs and classes that were once high-paying or high-status have seen dramatic drops in power, such as landed aristocrats before the industrial revolution, armored knights before guns, typesetters, weavers, and many types of skilled artisans. There is no law saying the same can’t happen to today’s high-status professions.[1]
Beyond income, wealth distributions could shift dramatically as assets like stocks and real estate are repriced to reflect this new world.[2]
For many people with above-average intelligence, their cognitive ability is a source not just of higher income but of status and self-worth. If this is you, you should think about how you’re going to handle your sense of worth in a world where your intelligence is no longer specially valued.
The more uncertain you are about the future, the more you should expect to end up near the middle of the income distribution. For an American, this means shifting your expectations towards an income of $30k/year post-tax.[3] But many in high-income countries like America underestimate how much further there is to fall. The more change we should expect, the less informative American income becomes as a baseline, as America’s historical sources of economic power (such as high skilled labor) are eroded, and the new value creators in society (AIs and the hardware running them) can move to the lowest-tax countries more easily than skilled humans can. The world median income is only $3,500/year, and the more change you expect, the more you should expect your income to drift closer to the world median than where it is now.
And uncertainty shouldn’t stop at income. The more we expect AI to change the world, the more we should widen our reference class in other ways as well, such as the geopolitical order.
Reversals of fortune are the norm throughout history: the relatively rich countries in 1500 are now relatively poor. Qing Dynasty China, the Ottoman Empire, native Americans, and many more cultures were all once dominant in their region, only to be surpassed and forced into irrelevance (or much worse) by upstart powers with more advanced technology. While America is currently the preeminent global power, this has only been the case for ~2% of recorded human history.[4] Most people don’t live in the world’s most powerful nation, so the more we increase our uncertainty, the less we should expect our home country to hold that position. And the more we rely on historical base rates, the less confident we should be that the current abnormally low rates of war and high rates of democracy will hold.[5] More broadly, the concept of the modern nation state is itself historically rare and unusual. There’s no guarantee it will remain the dominant form of political organization in the future.
We can continue expanding our reference class past all current humans, the more uncertain we are about the future.
The median human throughout history lived a life worse than the median human today; life expectancies used to hover around 30 years. And the human species itself occupies a rare position: humans’ level of dominance over our environment is unparalleled in Earth’s history, with our closest rival likely being the cyanobacteria that drastically altered the composition of the atmosphere 2.4 billion years ago. This is a rare position for any species to be in, and humans have only held this position for less than 0.01% of the history of life on Earth. AIs will soon have advantages over humans in intelligence, speed, and ability to quickly reproduce, and will likely eventually become dominant actors on Earth, as humans voluntarily hand AIs more power, or by outcompeting humans. Losing the spot as the dominant actor might not be pretty for humans: the median animal lives in Malthusian competition, and humans have historically used their dominant position as a species to do terrible violence to other species. Widen the reference class further still, and the Fermi paradox suggests that life itself is rare and atypical in our universe.
None of this is to say we should necessarily expect humans to end up like the median species on Earth. The point is simply that the more radical you expect changes from AI to be, the wider you need to make your reference class for the future. And unfortunately, the wider that reference class gets, the uglier it gets.
It’s possible that the wealth generated by AI will be so enormous that everyone ends up dramatically better off than they are today. But this isn’t guaranteed, and the reference classes above should give us pause.
The world around us often feels normal, but by many measures we occupy an incredibly unique and rare position. If you make $80k a year post-tax, you are in the richest 0.3% of the world, in a world richer than it has ever been, as a member of the most dominant species in the history of Earth.[6] The more radical you expect changes from AI to be, the more you should expect that uniqueness to fade.
Vernor Vinge, one of the original theorists of the singularity, describes it as an “opaque wall across the future”. Since you should loosen your expectations for the future the more change there is, and the singularity is the point at which change becomes infinite, then the singularity is the point at which all predictions break down, and we can say nothing about what comes afterwards.
I don’t know if we’ll get a singularity as Vinge describes it. But I believe we’re headed towards radically accelerated change because of AI, and a singularity is useful to consider as the most extreme version of this.
If we can’t predict where we’ll end up, and our best guess is that our life will resemble that of a randomly selected human, then we find ourselves in a situation eerily similar to what the philosopher John Rawls called the “Original Position.” The Original Position is a hypothetical position where individuals choose the principles for their society from behind a “veil of ignorance”, meaning that they don’t know which position in society they’ll occupy, and will be randomly assigned a position. Rawls argues that from this position, people would choose a strongly egalitarian social structure (though there’s plenty of debate on what exactly people would choose). Similarly, if we expect our lives to be as good as a randomly selected human, then we should want to structure society such that a randomly selected human has as good a life as possible.
I generally agree with Rawls that being behind a veil of ignorance makes you favor more egalitarian societies. However, being behind a veil of ignorance doesn’t necessitate going full hippy egalitarian. There are strong reasons for things like property and forms of inequality. Our uncertainty around AI doesn’t fully prescribe political positions. But it does demand, at minimum, looking the future square on, thinking about what it entails, and thinking through how we’d want the world to be structured if we could end up as any randomly selected member of it.
Crucially, none of this relies on there being a full singularity: this is only the limiting case. Rather, the degree of Rawlsianism prompted should be proportional to the degree of uncertainty we have about the future. The more change we expect from AGI, the wider our reference class should be, the more we should act as if behind the veil of ignorance, and the more we should put in place the structures that Rawls’ original position implies.
The force of Rawls’ argument is that it doesn’t rely on the original position inspiring a kumbaya moment, where the veil causes participants to suddenly see that our differences are immaterial, and that we should all treat each other better. Rather, it relies purely on the selfish motives of the individual participants. Since they don’t know where they’ll end up within the society, they’re all selfishly incentivized to structure society so that they’ll end up okay. Many current questions about justice are, in a sense, luxuries for those with the power to dispense it. They are questions about how they ought to act, but with few personal consequences if they don’t act justly. Radical uncertainty changes this, as it forces you to think about what type of precedent you want to set and what type of world you want to create, knowing you might just as easily be on the receiving end of justice as the giving end. Taking seriously that the last shall be first has strong implications for how you treat the last right now. Realizing this uncertainty could prompt people to selfishly push for a more egalitarian and altruistic society (such as current pushes for universal basic incomes). Similarly, a country that realizes it may not remain the world’s hegemon might reconsider how it treats less powerful nations. And the same logic extends to how a dominant species treats other species.
While expectations that the rich will just get richer dampen these Rawlsian implications (by removing uncertainty and lowering downside risk for those in power), this effect isn’t overwhelming. Many of the well-off people in the future will likely come from the well-off classes of today, but there will likely also be significant turnover, as the assets of many (future wages, real estate, specific stocks) will be radically repriced. A dominant sense among many well-off people in Silicon Valley is paranoia that they’ll be left in the “permanent underclass”. If there is any moment in history that they might be willing to give up some power to hedge their downsides, it might be now.
In many ways, the world we face is more radical than Rawls’ Original Position. While his veil of ignorance only forced uncertainty over where one would fall within a specific country, we also face increased uncertainty about where our country will fall within world affairs, and where our species will fall within the ecosystem. These uncertainties suggest an extension of Rawls’ egalitarian principles to how the international system should be structured to care for each country, and how the ecosystem should be structured to care for each species.
This puts us in a radically unique and valuable position within human history. While Ralws’ veil of ignorance was originally just a thought experiment, the more extreme we expect change from AI to be, the more uncertain we become about the future, and the more our position comes to resemble that of the hypothetical veil of ignorance. Let’s hope we can put our time in this unique position to good use.
In fact, as AI seems to be on pace to be able to automate white-collar tasks before most real-world physical tasks, there might be a window where the most valuable skills are dexterous hands.
Modeling by Epoch AI suggests that the wages earned by the remaining humans still employed in the final months before full automation of all labor would earn more in those few months than they did in their entire career combined (before their wages also collapse to zero when labor is finally fully automated).
This number is post-tax, per person (so also including children) income, so might seem slightly lower than common income figures.
Kind of sad that it's probably needed, but if this is a way to get more people thinking more empathically, we need to start sharing this idea.