The most important things that I want to say through this post:
JET (justice, equality, truth) is a decentralized governance and coordination system.
The goal of the system is to make people take the actions that maximize cumulative utility of the entire collective.
JET can be used to achieve coordination with regards to artificial intelligence risks. For example, JET can be used to:
In other words, JET is a strategy profile - an instruction about what each person should do. It also aims to be an equilibrium. JET also relies on a software.
Why don't people take actions that are the best for the collective, without any system:
JET aims to solve those two problems while minimizing maintenance costs of the system.
In the context of this paper, when I say "people", I don't necessarily mean "humans", it can include also other agents, for example artificial intelligence.
What is missing from the existing governance/coordination systems?
I believe that there are mostly 4 things that are missing in how the world is currently governed/coordinated:
For example, if we wanted to introduce a pause on building superintelligence, then this is how the above problem can come into play:
The rules are not decided based on aggregated information, but elected representatives. Those representatives are not aware of all problems with superintelligence because they don't have knowledge of the whole world. Therefore, they won't always make the best decision whether such pause should be introduced or not, and if it should, then what should be the specific rules.
In our existing system, everyone can contribute information about who complies and who disobeys the existing rules.
But people don't always have incentive to do that.
For example, if an AI company disregards certain rules regarding AI safety, then an employee of an AI company might inform appropriate institution about that. But they might face retaliation from their employer, so it's not necessarily in their interest to do that.
If an employee of an AI company was sufficiently rewarded for contributing such information, then it would still be in their interest to contribute that information.
The agreed rules are often imperfect. The bad actors often try to find ways to hack the system so that they can do something bad while not being held accountable for that. The good actors, on the other hand, could contribute some useful public goods for example, but they lack incentive to do that.
Consider the following analogy. It is known that reinforcement learning agents involve in reward hacking - instead of maximizing what humans want them to maximize, they sometimes maximize reward (which is an imperfect proxy for what humans want) without maximizing what humans aim to maximize through those agents.
In a similar way, the rules in our world aim to reward people for doing good things for the collective, and penalize for doing bad things for the collective. However, the rules are imperfect, and people sometimes involve in "system hacking" - trying to get the reward or avoid the penalty without doing what is actually good for the collective.
For example, a car dealer might sell a car that is worth $10 000 for $20 000. The economical system assumes that if a car is worth $10 000, then people will be willing to pay $10 000 for it. But in practice, the car dealer can use some negotiation techniques to convince someone that the car is worth more than it actually is, and get additional $10 000 that the car dealer doesn't deserve.
In case of a pause for AI research and development, an AI company might potentially find some loophole to keep doing the research or development despite the pause.
The rules are imperfect for various reasons, for example:
Imagine that there is a room with 5 people, and one person has a gun. There's nobody outside of the room, the world consists only of that room.
In such situation, it doesn't matter what political or economical system the people agree to, because the person who has a gun can force everyone else to do what they want, using the gun.
It is only possible to agree and enforce fair rules when people are reasonably equal in terms of their bargaining power.
However, without any norms, agreements or systems, the world tends to go towards greater inequality because:
The governance/coordination system needs to ensure that inequality will not increase to a level where some people have a lot of undeserved power while others have very little power compared to them.
In our world, there are already some ways to do deal with this, e.g. a combination of taxes and universal basic services.
JET consists of 3 components: justice, equality and truth.
The purpose of justice component is to motivate people to take actions that are beneficial to others and avoid taking actions that are detrimental to others.
In our current world, justice is solved by monetary trade and legal system. JET aims to solve that in a better way.
The purpose of equality component is to ensure that people won't end up being very poor, while others are very rich. There is a logarithmic relationship between utility and wealth (utility = log wealth). All else equal, it's better when wealth is split evenly.
In our current world, equality is solved by universal basic services. JET aims to solve that in a better way.
Justice and equality are in conflict. Rewarding people for good things that they did makes people unequal because the rewarded people become more wealthy. The goal is to achieve the right balance between justice and equality.
The purpose of truth component is to ensure that people can know the truth about what they need to know.
In our current world, truth is solved by prediction markets and justice system. JET aims to solve that in a better way (also using prediction markets but using them in a way that they are not used currently).
The purpose of justice component is to motivate people to take actions that are beneficial to others and avoid taking actions that are detrimental to others.
That can be achieved through a form of reciprocity - rewards and penalties.
There are two forms of reciprocity:
I propose that justice should be achieved using indirect reciprocity.
Because indirect reciprocity is superior in the following way. In case of direct reciprocity, if you don't meet with some person again, then there is no incentive to be good to each other. Because you will never meet that person again, so they won't have an opportunity to reciprocate.
Indirect reciprocity fixes that because if you treat the other person badly, then even if you don't meet that person again, then someone else will treat you badly.
Indirect reciprocity has also other advantage related to equality that I will talk about later.
Let's categorize reciprocity in the following way:
All kinds of reciprocity have their pros and cons.
With agreed reciprocity, there is clarity about rules, everyone knows what is good and bad. Therefore, there is more psychological safety.
Additionally, the rules are the result of a process designed to come up with good rules. Therefore, agreed reciprocity results with a better answer to what is good/bad than if a single individual decided it by themselves every time when they take an action.
Additionally, unagreed reciprocity requires knowing intentions of a person. It's hard to know someone's intentions.
The rules are often imperfect.
If the actions are reciprocated based on agreed rules, then people can find loopholes in those rules. They can find a way to intentionally harm someone else without disobeying the rules. Or they can find a way to get rewarded by those rules without creating any good.
With semi-agreed reciprocity, if it's clear that someone intentionally did something detrimental to others, then they will be penalized, which removes the incentive to find loopholes in the existing rules. Analogically, if someone intentionally did something intentionally beneficial to others, then they will be rewarded, which creates the incentive to take those actions, even if they are not rewarded by the rules.
Unagreed reciprocity can be useful when semi-agreed reciprocity is not possible for any reason.
Since all kinds of reciprocity have their advantages, I propose that both should be used to certain extent. The people should agree on rules (in later sections, I will describe how people can agree on those rules). But to some extent, they should also reward intentionally beneficial actions, and penalize intentionally harmful actions.
Semi-agreed reciprocity and unagreed reciprocity allows the system to be partially foolproof to imperfect rules.
Our existing system relies almost completely on agreed reciprocity and to some extent on unagreed reciprocity (mostly direct). Semi-agreed reciprocity is rather not present, perhaps we should use it more.
The following questions have not been answered in this section:
Those questions will be answered in the other sections.
Imagine that you play a Monopoly game. Each monopoly game follows the following trajectory:
Humanity is likely to follow a similar trajectory - the rich get richer and the poor get poorer, IF humans are self-interested and there is absence of norms, laws or systems that would prohibit such scenario.
That is because of the following properties of the human world:
[^1] This is not limited to taking the resources in a physical sense.
High inequality is detrimental to cumulative utility (long-term happiness) of all people because of diminishing marginal utility of resources and power. If we assume that , then, all else equal, greater equality is beneficial.
That inequality also leads to end of democracy, because the poor people lack ability to establish and enforce fair rules. Rich people can use their resources and power to create a world that is favorable to them.
However, the difference between monopoly game and human world is the goal of the game. In monopoly, the goal is to eliminate all other players. In human world, the goal is to maximize your utility (long-term happiness) which is roughly equal to .
Therefore, monopoly is a zero-sum game - it's not possible for everyone to win. Human world is not a zero-sum game.
Let's think about what would be the optimal strategy for monopoly, if the goal was to maximize , instead of eliminating all other players. Let's suppose that the players are equally good at game. So, the game would depend solely on luck.
The increasing disproportion in wealth would be undesirable for an average player. So, the optimal strategy would be for players to make an agreement that whoever gets lucky and becomes richer shares their wealth with other players.
The agreement could be enforced by agreeing that the wealth will be shared only with the players who share their wealth with others. If a player stops sharing wealth, then the other players stop sharing wealth with that other player.
The game would last practically infinitely with all players being equal.
The same thing could be done in human world. We can make an agreement where people share resources with those who have less resources. If they don't contribute their resources, other players don't share resources with them when they are in need.
We can view this as a form of wealth insurance.
Let's suppose you buy insurance for not being able to work due to illness. When you buy it, you know that you will most likely lose money, because most likely you won't get any illness. But you buy it because you know that in the event of illness and not being able to work, you will need money significantly more.
Majority of people will never get sick, and their money is used to pay insurance to the minority of people who got sick.
In the same way, in the agreement proposed above, people agree to share their wealth with the people in need. Most people will lose their money, but they still have an interest to do that, because if they happen to be in need, they will need it more, due to diminishing marginal utility.
The above idea works proportionally well to the amount of uncertainty about the power distribution in the future. The wealthy people have interest in contributing their resources only as long as there is a chance that they will lose their power to others.
But, as we've established above, without any norms, agreements, systems or altruism, humanity is likely to follow the standard monopoly trajectory (winner takes it all), so some level of uncertainty exists. A person who is currently rich can for example lose their wealth to another rich person who is the first to build superintelligence and use that superintelligence to steal resources from all other people (assuming that the AI alignment problem is solved).
In our world, people are not at a stage when all people are equal (like at the beginning of the monopoly game), therefore the above agreement wouldn't lead to complete fairness, but it's possible to make an agreement that doesn't lead to complete fairness that is still beneficial to all participants.
The above-proposed agreement can be simply a rule that people agree on using JET system. In the next section, we will discuss how the rules can be established and enforced (including this agreement).
When I say "AGI", I refer to Artificial General Intelligence. Pre-AGI world means the world before transformative AI, and post-AGI world means the world after transformative AI.
We can categorize the resources in the following categories:
In pre-AGI world, the bargaining power is greatly distributed. That is because weapons (like a rifle) require humans to operate them, therefore concentration of military resources is not possible.
Additionally, people have different skills that are needed by other people, nobody has the skill to do the work of billions of people, so people are dependent on each other.
In pre-AGI world, imaginary resources (like money) have value, because there is utility in them having value - if people stopped assigning value money and equity, our political and economical system would collapse and people couldn't be rewarded for their work. As a result, everyone would end up worse.
In post-AGI world, skills don't matter (because AI can do execute the work that previously required skills).
Military resources can potentially be concentrated in the hands of a small number of people because autonomous weapons don't require a large group of people to operate them.
Tangible resources can be forcefully taken by those who have military power. In pre-AGI world, people can also use military power to steal tangible resources, but firstly in pre-AGI world military power is less concentrated (because it requires humans to operate weapons), and secondly in pre-AGI world people in power need the system to work.
On the other hand, in post-AGI world, If artificial intelligence is very powerful, to the extent that whoever has operational ability over it (and the autonomous weapons), that person or group of people has control over the world to the end of their life, then those people might no longer have incentive to sustain the existing political and economical system.
The reason why people need the existing political and economical system is that it gives them safety that they will be protected by it, when they lose their power. But when the power is so strongly concentrated that the people in power have guarantee that they will be in control to the end of their lives, then they might no longer have incentive to sustain that system.
For that reason, according to my world model, military resources are the most important resources in post-AGI world. In order for the world to remain equal, those military resources must be either equally distributed (to a reasonable extent), or the world must simply refrain from using autonomous weapons to avoid concentration of military resources.
The first option comes with a cost that if all people are in the possession of weapons, then they might use it to commit crime. For that reason, the second option (banning autonomous weapons) is better.
Since everyone can fall a victim to concentration of power, banning autonomous weapons would be in the interest of everyone.
The purpose of truth component is for people to be able to know what the truth is. The purpose of truth component is to aggregate the information from all people, and decide what is true based on that aggregated information. The Justice and Equality components need to rely on this component as a source of truth.
Justice component needs to know:
The idea is that people use the truth component to decide what rules would be good, and the rules that have been decided to be good, simply become the rules, and they are enforced through reciprocity (see "Justice" section).
The people also contribute information about what is needed to the Truth component.
In order for the rules to be enforced through reciprocity, people need to know who complied with the rules and who didn't, and again - people use truth component to accomplish that.
Equality component needs to know:
Truth component can simply be a facilitated through a prediction market. In it's beginning stage, it can also be an Internet forum, for simplicity.
In both cases, it needs to be fully decentralized. Otherwise, the people who are in charge of the system could manipulate it for their own advantage.
The system can be facilitated through a locally-run, open-source software that communicates with the software of other users through a peer-to-peer network. The software has to be locally-run, open-source and communicate through a peer-to-peer network because if there is a central server that runs the software, then the people in charge of that server would have too much power, making the system fail at its purpose.
Truth component rewards the people who contribute information to it. In a standard prediction market, when people contribute new information, then they make money. In this case, I propose that people are rewarded simply with "reciprocity points", i.e. if someone is right, then it is recorded in the truth component itself that this person did something good, and therefore deserves positive reciprocation.
It is also expected that some information in the prediction market would be contributed in an automated way.
In this paper, I won't go into how such decentralized prediction market or decentralized forum could work in detail or on a technical side, I leave that as an unanswered question (I have some idea, but I won't have time to go into that).
JET is not supposed to be based on money. But money plays important functions in the existing system, specifically:
Therefore, it's worth asking: how does JET realizes those functions without money? And what is the motivation behind that?
In JET, people are rewarded with reciprocity. If a cleaner cleans a house, that information is recorded on a decentralized prediction market / forum, and the prediction market / forum updates information about how worthy of positive reciprocation the cleaner is. The prediction market / forum can store information about some "social score" of the cleaner which is an information about how much value the cleaner contributed to society vs how much value they received. That social score gets updated, when the cleaner cleans the house.
As for the information about what people need, the people simply contribute the information about what they need to the Truth component (decentralized prediction market / forum).
A JET equivalent of a person A buying milk from a person B for $8 would be:
If a person A tries to abuse the system and contribute information that they need things more than they actually do in order for people to give them those things, then the prediction market can learn that the person A tends to abuse the system, and the person A will end up being penalized.
Let's suppose that a person sells a car to another person. Later, it turns out that the car wasn't worth the price because the buyer wasn't aware of some problems with the car.
The traditional system doesn't offer any solution to it. Once the transaction was made, people can't update the information about the value of the contribution (in this case, giving a car to someone). There are some rules that could prohibit cheating in some way, but those rules are imperfect and there are no rules for every possible way that the seller could cheat the system.
In JET, people can update information about the value of the contribution. If some new information surfaces, and if confirmed by a third person, the Truth component can update its knowledge about the value of the contribution, resulting in a fairer system.
In JET, the value of negotiation skills is therefore lower. That is good because negotiation is a zero-sum game. It's better for the collective, when people focus on playing positive-sum games.
Generally, one of the most important differences between JET and existing systems is as follows.
In existing systems, if someone manages to cheat the system, then the rules might be improved later to avoid such situations in the future, but the person who cheated the system still benefits from that.
In JET, when someone cheats the system, then if new information comes to the surface and it becomes clear that the person intentionally cheated the system, the fairness is recovered. Therefore, the person has lower incentive to cheat in the first place.
JET solves the problems that I have discussed at the beginning of the paper. Specifically:
The biggest limitation of the system is that assuming that people are self-interested, once people are unequal in terms of their power, it can't be used to create a perfectly fair system. The mechanism for achieving equality works only if there's uncertainty about future distribution of power.
However, if we assume that artificial intelligence is a transformative technology that can potentially concentrate power in the hands of the few and make people live very long, then the incentive to participate in the described agreement to ensure safety is strong, so a lot of equality can be achieved.
Robin Hanson introduced Futarchy which is also a governance/coordination system.
The commonality between Futarchy and JET is that both systems propose to create rules based on the aggregated information from all people instead of the rules being created by the elected representatives. Both systems propose to rely on a prediction market for that purpose.
The differences between Futarchy and JET are as follows:
