Summary of key arguments

At present, there are no rules around the creation of artificial sentient beings. Anyone can create them, own them, make them do whatever they want, and treat them however they want to. 

This is bad, and could lead to a lot of suffering for these beings. 

The creation of artificial sentience ought not be left as an unregulated free-for-all. It ought to be regulated by governments, on behalf of society.

The moral stakes are so high that we should wait before creating artificial sentience, rather than rush into creating it, whether deliberately or accidentally. Very possibly, it may be best not to create it at all. 

Responsible researchers and companies should not seek to deliberately create artificial sentience.  

And governments should take steps now to prevent the creation of artificial sentience, at least in the short- and medium- term. 

Even if this proves difficult or impossible, we should definitely ban the creation of artificial suffering. 

In future, if we decide to permit the creation of artificial sentient beings, it should be carefully regulated, in order to protect the interests of these potentially vulnerable beings. 

Much more work is needed to figure out exactly how to implement this approach, and make it work in practice. Readers of this piece should contribute to this process. 

Introduction

We will, collectively, as a society, need to figure out how to deal with the potential creation of artificial sentience. Through the political process, societies and their governments will need to decide whether they want to permit the creation of an entirely new class of sentient beings, and how they want to regulate this. We’ll have to answer questions such as: 

• Should we create artificial sentient beings?
• Who should be able to create them? Anyone who wants to?
• If they are created, should there be some rules about how they are treated? Who should set these rules, and what should the rules be?
• Should the creation of certain types of artificial experience be permitted, or not? Might we want to prevent the creation of artificial suffering, but allow the creation of artificial happiness? 

It feels like our options fall into a few broad categories:

1. A free-for-all: no rules or regulations on the creation of artificial sentience
2. Voluntary codes-of-conduct
3. Government-mandated regulation
4. A ban - temporary, or indefinite - on the creation of artificial sentience, or of specific types of sentience. 

This post is an initial, incomplete attempt to start to flesh out some of these options, and to set out the pros and cons of each of them. 

Context 

There is massive uncertainty about whether artificial sentience is possible, and how we will know whether a given system is sentient. Much more research is needed into these foundational questions. But, we should also start thinking now about how to regulate. We can't wait around for all of the philosophical questions related to consciousness and sentience to be solved - if they ever will be. The rapid pace of AI development means that this is now a practical, objective issue in the world today, that we can and should engage on.

There are probably two broad types of actors who might create artificial sentience:

1. Researchers who are deliberately trying to create it. (Here’s one example. Some leading consciousness researchers think it would be “monumentally cool’” to create artificial consciousness. Others are even deliberately exploring how to create ‘pain’ in artificial beings.)
2. Leading AI labs who aren’t trying to create artificial sentience, but who are building complex AI systems that might end up being sentient. 

And we’ll have to think about two things: 

1. Research which could lead to the creation of artificial sentience
2. The actual creation of artificial sentience. 

(In addition to this, there are some areas of biological research that may end up creating novel forms of sentience. For example, brain organoids, Dishbrain. I won’t cover these in this piece, though some similar considerations may be relevant.)

This piece will avoid going into detail on the practicalities, science and philosophy of this field; it will just focus on some high-level considerations around the broad policy options for our approach to the issue. 

Pros and cons of different broad options for the regulation of artificial sentience

We will now run through some of the broad main options for our approach to regulating the creation of artificial sentience, and examine their pros and cons. 

There are two broad classes of things to think about here. Firstly, what are the ‘objective’ pros and cons of each option for regulation, as an end state - assuming we can actually get to that end-state. Secondly, what are the tactical pros and cons of arguing for, and lobbying for, each option. For example, we might think that voluntary codes of conduct are a sub-optimal option, but that they will be tactically easier to achieve in the short-term, so we should start by arguing for them. 

We’ll start by looking at the ‘objective’ pros and cons of each option for regulation, and then, later in this piece, consider some thoughts around tactics. 

Option 1: Free-for-all

Definition 

There are no rules, regulations or laws around the creation of artificial sentience. Anyone who wants to try to create an artificial sentient being can go ahead and do so. They don’t have to apply for any kind of permission, or abide by any rules or guidelines. Anyone can do anything they want. Anyone can do any type of research, build any type of system, with any chance of it being sentient, with no rules around this. 

Anyone who succeeds in creating an artificial sentient being - whether deliberately or not - can then treat them however they like. They can do whatever they want with them. They can make the artificial sentient beings do whatever they want. The people who create these beings will be the owners of the artificial sentient beings. The beings will be the property of their owners. The owners can cause them pain and suffering - whether deliberately, or inadvertently - with no consequences or restrictions. 

This is the current state of affairs.  

Pros of a free-for-all

• Accelerates the creation of artificial sentience. Having zero rules and regulations probably means that we will create artificial sentient beings as soon as possible. If we believe, for some reason, that these beings will generally be treated well, and enjoy lives that they are happy to live, then we might welcome this: any restriction or delay would make the world worse off. Let’s just go ahead and create them, and trust to the goodwill of their creators that they will be treated well and everything will turn out fine. 
• General benefits of unrestricted AI development. AI advancement plausibly brings big benefits for addressing various problems, and insofar as restrictions on artificial sentience also restrict AI advancement, this could be bad.

Cons of free-for-all 

• Potential for a gigantic moral catastrophe. Leaving the creation of artificial sentience totally unregulated may lead to a gigantic moral catastrophe, and massive amounts of suffering. Looking at the historical human treatment of other sentient beings - such as ‘out-group’ humans, and animals - suggests that there is the chance of us doing serious harm to any new class of artificial sentient beings. Given the possibly astronomical number of such beings, this could be very, very bad, and create a big S-Risk - suffering risk. The beings that we create will likely not look like us, and may not trigger the empathy-driving pathways that tug on our heartstrings and elicit compassion. As with factory farming and animal experimentation, commercial incentives may incentivise doing harm and causing them to suffer - even without any bad intent from us. We might not know if what we’ve created is sentient or not - so we might not even know that we are doing bad things to them. Although it seems highly unlikely that current AI systems are sentient, the way we treat them at present would be bad, if it were repeated on potential sentient systems in the future. Bostrom and Shulman (2023) write: “training procedures currently used on AI would be extremely unethical if used on humans, as they often involve: no informed consent; frequent killing and replacement; brainwashing, deception, or manipulation; [...] routine thwarting of basic desires; for example, agents trained or deployed in challenging environments may possibly be analogous to creatures suffering deprivation of basic needs such as food or love; [...] no oversight by any competent authority responsible for considering the welfare interests of digital research subjects or workers.” They go on to argue that “as AI systems become more comparable to human beings in terms of their capabilities, sentience, and other grounds for moral status, there is a strong moral imperative that this status quo must be changed.” 

Concluding judgment 

The current status quo, of zero rule and regulations around the creation of artificial sentience, is terrible, and indefensible. It could lead to a moral catastrophe. It seems very bad to bring into the world a new category of sentient beings, in a totally unregulated way, which enjoy zero legal or other protection. 

We should take action. The creation of artificial sentience ought to be regulated in some way. 

Potential next actions 

People who agree that the current unregulated free-for-all is bad, should engage on this issue, and help to flesh out, and deliver, options for regulation. 

Option 2: Voluntary codes-of-conduct

Definition 

Companies and researchers might sign up to voluntary, self-imposed rules or ‘codes of conduct’ around the creation and treatment of artificial sentience. 

Pros of voluntary codes-of-conduct. 

• Short-term feasibility. In the short term, voluntary rules may be the most feasible step. AI labs, and consciousness researchers, are much more interested in, and expert on, this area, than government officials. They can take action immediately to help safeguard against a moral catastrophe. Artificial sentience protections could become part of the real, live, AI safety debate and process that is underway at the moment - e.g. relating to Responsible Scaling Policies.
• Establishes the credibility of artificial sentience as a serious area of concern. In the short term, a light-touch, voluntary approach may help secure buy-in from leading AI labs and researchers. This could be an important first step in terms of establishing the legitimacy and credibility of this nascent field.
• Offers some protection. Even without legal enforceability, such policies may provide some protection for artificial sentient beings, and make it more likely that companies will act in accordance with their stated principles.
• Creates peer-pressure for higher standards of protection for artificial beings. Companies which sign up for voluntary actions may become allies in pressuring other companies to sign up too, to keep a level playing field. 

Cons of voluntary codes-of-conduct. 

• Offer very little real protection. Companies have an incentive - perhaps even a legal duty - to maximize shareholder value. The interests of shareholders may not be exactly the same as the interests of the artificial sentient beings that the company owns. Thus, voluntary codes of conduct are unlikely to offer adequate protection. And of course, this approach still allows the development of sentient AIs, which may lead to astronomical suffering.
• Optional. Less-cooperative actors may just opt-out and benefit from a competitive advantage.
• ‘Sentience-washing’. This approach may confer upon companies a public relations benefit, without adding much genuine protection for artificial sentient beings - ‘sentience-washing’, like ‘greenwashing’.

Concluding judgment 

This approach is probably a good place to start. Bostrom and Shulman (2023) give some thoughtful tactical reasons for wanting to start with this sort of approach towards this still-nascent field. 

But, I claim that it’s unlikely to be where we want to end up. To guarantee the protection of the interests of artificial sentient beings, we will need much more than purely voluntary action by the most responsible actors. We will need proper regulation, with teeth. 

Potential next actions

• Companies might start thinking about drawing up voluntary codes of conduct on this issue.
• Other interested actors might start thinking about exactly what such codes of conduct should contain, and lobbying for their adoption. 

Option 3: Government-mandated regulation

Definition 

Mandatory, government-enforced rules which govern who may create sentient AIs, what types of beings can be created, how they may be treated, etc.

John Basl and Eric Schwitzgebel have argued for regulation - the creation of ‘oversight committees’ to decide what research should be permitted. Perhaps some version of this should be government-enforced. 

An incomplete/partial initial list of pros and cons of this approach:

Pros of government-mandated regulation 

• Could provide meaningful safeguards against moral risk, whilst still allowing the creation of beings with positive welfare. 
• Limits the number of organisations that may create sentient AIs, making monitoring easier.

Cons of government-mandated regulation

• Still allows the development of sentient AIs, which may lead to astronomical suffering.

Potential next actions

Extensive further thought is needed on this, to work up concrete options and consider the pros and cons more deeply. The preliminary steps for such work should start now. 

Option 4: Banning the creation of artificial sentience (permanently, or temporarily) 

Definition 

Thomas Metzinger has argued for a 50-year moratorium on the creation of artificial sentience, “strictly banning all research that directly aims at or knowingly risks the emergence of artificial consciousness”. 

Pros of preventing the creation of artificial sentience 

• May decisively prevent huge amounts of suffering. If we turn history down the path of "no artificial sentience gets created", we avoid all the potential astronomical net-negative scenarios. This could be the single thing which prevents the largest amount of suffering, ever. This feels like a really big and important consideration. A lot of people, rightly, worry about S-risks. A world in which artificial sentience simply isn’t developed, is a world where the worst S-risks can never happen. Preventing the development of artificial sentience might be the single best thing ever, morally, and save huge numbers of beings from huge amounts of pain and agony.
• Gives us time and option value. We can always lift the ban later. Perhaps after a ‘long reflection’, we’ll figure out a way to guarantee that artificial sentient beings will be treated well, and live happy lives. We can then reverse the ban. This doesn’t apply the other way around - it’s harder to put the genie back in the bottle, rather than to prevent it getting out in the first place. It would have been better, for example, to have prevented factory farming from having been developed, rather than to try to fight it once it had been developed and proved economically advantageous.
• Deontological reasons. We might think it's simply deontologically wrong for a private individual or company, or a government, to be able to create and own another sentient being. Therefore, we should try to prevent this from happening.
• Potential benefits for AI safety. If we build sentient beings, we may decide that we should grant them rights and respect their interests. In some circumstances, maybe this could be bad for AI safety. If we simply don’t build sentient AIs, we don’t have this problem.
• Frees up time to work on other priorities. If we can successfully ensure that artificial sentience is not created, it will free up the time of altruistic people who can simply concentrate on the sentient beings that do exist, rather than having to worry about a vast new class of artificial sentient beings.

Cons of a ban

• We will miss out on the benefits of artificial sentience. If we don’t create artificial sentience, there might be a lot of happy sentient beings who would never be created. Depending on one’s view of population ethics, this could be very bad.
• Risks from partial enforcement. If not fully enforced, a ban might just hold back the most ethically-concerned labs/institutions; thus, by definition, the least-ethical institutions will be the ones who create artificial sentience. Similarly, maybe a ban would only be enforced in certain countries. Sentient AIs may be produced in other countries first, and/or only, where we might expect them to be treated less well. In addition, perhaps companies or researchers may work underground, secretly, on the creation of artificial sentience.
• We might miss out on the benefits of related research. If we go for a fairly broad ban on research which could lead to the creation of artificial sentience, this might hold back progress on a bunch of issues that would otherwise have provided benefits for society.
• Encouraging cover-ups. If creating digital sentience is illegal, then a company or researcher who suspects they might have created artificial sentience may be unwilling to disclose this, and unwilling to take actions that could improve the welfare of the artificial sentient being, because they may face penalties if they ‘confess’ to having created sentience. 

Concluding judgment 

I find the idea of simply trying to prevent, or at least delay, the creation of artificial sentience very attractive, due to the force of the arguments in favor of buying us time and option value, and preventing astronomical suffering. 

One's foundational ethical views, and views on population ethics, will likely play a significant role in shaping one’s opinion on this. Those who are more focused on minimizing suffering might view the prevention or delay of artificial sentience as a favorable option. In contrast, a classical utilitarian who believes that artificial sentient beings will, on balance, likely have lives worth living, and who believes we should create lots of these beings, might be more skeptical about efforts to prevent the creation of artificial sentience. 

However, I argue that there are strong reasons to advocate for *delaying* the development of artificial sentience, even for those who are most excited about the potential of artificial sentience to lead to astronomical quantities of joy. 

If we do end up creating artificial sentience, it seems really, really important to ensure that the development of it goes as well as possible, with the strongest possible ethical safeguards in place, for the benefit of human society, and for the artificial beings themselves. To achieve this, we should pause and draw breath, before rushing headlong into creating artificial sentience. At the very least, a period of pause, during which we can organize and prepare thoroughly, feels essential before we take this momentous moral step. 

In the grand scheme of things, over the lifetime of the universe, a delay of a few decades or so is unlikely to significantly reduce total utility if it helps ensure that everything is done perfectly when we finally move forward. Plowing ahead with the current, unregulated free-for-all seems like a potential recipe for disaster and is certainly not the optimal way to approach this monumental development.

Thus, we should support, for the short- and medium- term, a ban on the creation of artificial sentience - at least until we figure out how to ensure everything goes well, if and when we do choose to create it. 

Potential next actions 

People interested in this should work up the details of how a ban would work, and campaign to get it enacted. 

Option 5: Banning the creation of artificial suffering   

Definition 

We could ban the creation of artificial suffering - ‘negative valence’ in artificial beings. It would be permitted to create artificial sentient beings with neutral or positive affect, but not permitted to create beings whose experience is dominated by suffering. 

Bostron and Shulman (2023) propose that: “to the extent that we are able to make sense of a “zero point” on some morally relevant axis, such as hedonic well-being/reward, overall preference satisfaction, or level of flourishing/quality of life, digital minds and their environments should be designed in such a way that the minds spend an overwhelming portion of their subjective time above the zero point, and so as to avoid them spending any time far below the zero point.” 

I suggest that we should look to go further than this, ensuring that no time is spent beyond the zero point.  And we should have this actually mandated and enforced - rather than simply requested. 

Pros 

• Best of both worlds. Prevents a moral catastrophe, whilst allowing the benefits of creating artificial sentience. We may end up with lots of happy artificial sentient beings, and no suffering ones.
• Hard to oppose. While some actors may push back against a total ban on the creation of artificial sentience, it seems hard to imagine anyone arguing hard in favour of being allowed to create artificial suffering.

Cons 

• Info-hazard. Tactically, perhaps even talking about the concept of artificial suffering might make it more likely to occur, for example by raising the salience of the concept in people’s minds.
• Difficult to define. Predicting the overall experience of a being is a difficult forecasting challenge and a complex moral question. 

Concluding judgment

This feels pretty close to a no-brainer, to me. It seems clear that our default presumption should be to ban the creation of artificial suffering. 

Potential next actions 

We should work to bring about a ban on the creation of artificial suffering. There’s clearly a lot of work to do here, much of it quite fundamental and difficult. Interested actors should get to work on this.  

Tactical considerations 

As well as considering where we’d like to end up, in terms of regulation, we’ll also need to think tactically about the best practical course of action, in the world as it is today. 

Tactical advantages of advocating for a ban on the creation of artificial sentience 

There are some tactical advantages to making an ambitious, maximalist ask, such as for a ban on the creation of artificial sentience. These include:

• Potentially achievable. There are examples of where humans have successfully prevented or limited the development of certain technologies. It might actually work, particularly if it turns out that it's actually pretty difficult to create artificial sentience.
• Overton window/radical flank. If some actors advocate for a ban / moratorium, it makes milder regulatory proposals seem like the moderate, compromise option. Arguing for a big, maximalist goal - a total ban - may maximise the chance of getting some meaningful control/regulation, even if we don't succeed in actually getting a ban. Whereas if we start with a more minimalist ask, we might not even get that. If we assume that whatever we ask for will end up getting watered down, we might as well start with a big, maximalist ask, even if we don't think it's likely that we will actually get it.
• Simple and clear. Whereas regulation could shift and dilute, a clear and simple ask and red line - don’t create artificial sentience (or suffering) - may be easier to maintain.
• Public support. There may be public support for this. A survey by the Sentience Institute found that 69% of respondents said they would ‘support a global ban on the development of sentience in robots/AIs.’ Calling for a ban may resonate with a broad spectrum of opinion across society. 

Tactical disadvantages of advocating for a ban on the creation of artificial sentience

• Provokes a backlash. Advocating for a ban may alienate AI researchers, and companies, who may resent attempts to stop their work. It may provoke an immediate, knee-jerk hostile reaction from them - including attempts to ridicule and denigrate the entire concept of artificial sentience. They may be joined by people with a generally strong anti-regulation gut intuition on policy matters.
• Too early? Advocating for a ban now, when it’s arguably clear that there are no sentient AIs yet, may be too early, and risk a lack of credibility. It may fix negative perceptions because of the 'obviousness' that current systems don't deserve any sort of protection. Further, at this point, the people arguing against a ban may have more power and influence than those arguing in favor of it. Perhaps we should build a community of credible, pro-protections-for-artificial-sentient-beings advocates first.
• Difficult to define. It’s likely to be difficult to define precisely what would be in-scope for a ban. Concepts like 'sentience' and 'consciousness' are already very difficult to define and identify. The broader the definition and the more cautious you are, the more research that would be held back and the more people the regulation would anger.

Conclusion 

I think the principle that we ought not - at least not yet - to create artificial sentience, is a powerful and important one. 

The arguments for it are strong. 

It seems potentially achievable. 

It passes a common-sense gut check. The idea that we ought not to create - or at least *rush* to create - artificial sentient beings, feels pretty normal and sensible, and not a particularly wild or out-there ask. 

It may command wide support.

I think it’s valuable to start spreading this meme more widely, scrutinize and debate it, and building a field of engaged people working on it. 

In terms of immediate next steps, I have sympathy for Bostrom and Shulman’s arguments in favor of taking a calm, thoughtful, non-sensationalising, non-adversarial, non-polarising approach towards talking about this issue. I agree with them that it’s probably too early for  mass public outreach, or even, perhaps, lobbying of governments. 

But I also think we can and should be a bit bolder than they propose. I don’t think we should just assume that it’s inevitable that artificial sentience will be created. We should, at the very least, debate and question this assumption. And we should be clear that purely voluntary codes of conduct are not acceptable as an end-state, and that a ban or delay should be firmly on the table as a plausible policy option for debate and consideration. 

Potential next steps 

There is a huge amount of work to be done here. Specifically, potential next actions could include:

• A comprehensive mapping of all the labs/projects which exist in the world today, which could lead to the creation of artificial sentience.
• A comprehensive mapping of all the proposals for regulation/prohibition/voluntary action/guidelines related to artificial sentience research that have been put forward.
• A systematic assessment of the pros and cons of each of the different proposals, inviting a range of views, and using tools like double-cruxing to try to reach some well-grounded conclusions.
• An attempt to align on a set of policy proposals/asks, which we think will do the most good.
• Work up these policy proposals into concrete, actionable forms. For example, potentially, detailed drafting of processes/rules and regulations/legislation that could be introduced, detailed consideration of exactly which fields and types of research should be covered, etc.
• Taking action to implement these proposals. This could include:
  • Lobbying governments to enact legislation/regulation
  • Engaging with companies/researchers to adopt voluntary guidelines and standards
  • Contributing to public conversations on this issue.

If you are reading this and are interested in this topic, and minded to undertake further research and action, please comment below, and/or direct message me; I'd be happy to link you up with other people who are interested in this topic.