Comments
53
TLDR: Several US states are considering banning legal personhood for AI systems and/or declaring that AI systems cannot be conscious. The primary motives include religious beliefs, ensuring human accountability for harms, and child safety. We think both moves are premature. Declaring that AI cannot be conscious is especially unwise, since that is a scientific/philosophical question lawmakers cannot resolve. Legal personhood is a harder question with potential costs and benefits, and for now we believe lawmakers should avoid making a lasting decision.
Once the stuff of science fiction, the prospect of conscious AI is now being taken up by state legislatures across the United States.
Since 2022, bills have been introduced in twelve states that variously declare AI systems to be non-conscious, ban legal personhood for AI systems, or include related provisions. We call these the "Exclusion Bills." Four have already passed, in Idaho, North Dakota, Utah, and Tennessee. These bills represent the first legislative efforts to preemptively rule out AI rights and, in some cases, AI mental states. We investigated each of these bills and compiled the first comprehensive documentation of the movement in a new report. This post summarizes the key takeaways.
Our view: We share some of the concerns driving these bills. But we think they will do little to address the real risks, and that legislating away hard questions about consciousness on the eve of a technological revolution is unwise. On the question of legal personhood, we are unsure what the best approach will be (see discussion below), but we think that for now, lawmakers should avoid making strong policy commitments.
Read the full report here.
What is AI personhood, and why would one want to ban it?
Personhood is a legal construct that confers the ability to bear legal rights and duties. The clearest example is humans: we can own property, exercise freedoms such as speech or religion, and answer for crimes and civic duties. But “personhood” isn’t unique to humans. Corporations can also own property, sign contracts, and be held liable for harms, though they lack rights like voting.
Some scholars argue that corporate-style legal personhood may be needed for agentic AI systems, in order to incentivize them toward economically productive and safe behavior. Note, though, that one could argue that AI personhood would harm AI safety by enabling AI systems to legally accumulate resources and concentrate power. We are open to either possibility, and do not have a strong stance on AI personhood. Meanwhile, other scholars argue that stronger forms of personhood may eventually be required to keep the law coherent.
Alongside these legal debates is the emerging possibility of actual AI consciousness. As AI systems grow more capable, experts are divided about whether there might soon be “something it is like to be” one. Some argue that on the popular theory of computational functionalism, building conscious AI may be possible in the near future. Others hold that biology is strictly required for consciousness, making it very unlikely in digital systems. This division seems unlikely to resolve soon.
Uncertainties notwithstanding, if AI were to become conscious, this would strengthen the case for granting it legal rights, responsibilities, or protections against abuse.
Even as experts remain divided on the legal and scientific questions, politicians in several states are eager for definitive answers. In our report, we identify a cluster of motivations driving the Exclusion Bills:
Many Exclusion Bills share near-verbatim text, and our report documents ample evidence of coordination across state lines. Meanwhile, there has been virtually no organized opposition to the Exclusion Bills, and little input from experts on AI, consciousness or related matters.
In many ways, the Exclusion Bills are motivated by a commendable desire to responsibly manage the transition to advanced AI. Unfortunately, they needlessly constrain future policymakers while doing little to address the real risks.
Consider the case of child safety. Will blocking a lawsuit against an AI system prevent a child from forming an unhealthy attachment to ChatGPT? We doubt it. Declaring AI to be a non-person is unlikely to impact how children experience interacting with it.
Or take the concern about liability. If the fear is that AI companies will use their products as liability shields, the law already has ways to hold multiple parties responsible where appropriate. A blanket ban on AI liability does not sharpen corporate accountability, and if anything, it could discourage future policymakers from carefully determining who should actually be on the hook when the highly agentic AI systems of the future cause harm.
More importantly, these bills are premature. Whether an AI system could ever be conscious is one of the hardest problems in philosophy of mind, and the experts who study it do not agree. Declaring by statute that AI cannot be conscious does nothing to change the underlying facts, whatever they turn out to be, and to settle it by statute now reflects a superficial engagement with the question. What it does is set a precedent which may make these questions hard to revisit when they matter.
The motivations behind the Exclusion Bills are understandable, but the approach is misguided. We would rather see lawmakers admit where they face uncertainty while remaining ready to act decisively once the evidence warrants it. In the meantime, they might redirect their efforts towards policies that more directly address the most consequential harms from AI systems, such as by clarifying liability for AI harms or restricting children’s access to chatbots. When it comes to personhood and consciousness, the wiser course is to keep our options open.
