Cross-posted from my Substack.
Epistemic status: I’ve puzzled over this for a long time but I’m not sure I came to the right answers. I'm sharing this partly because it might help others work on resource allocation, but mostly because some smarter reader might tell me how to do it better.
TL;DR:
* I he...
With the AI windfall, mental health may be taken increasingly seriously by mainstream EA. There’s increasingly room to explore the cause area and answer some of the biggest uncertainties; at the same time, costs of the most effective interventions are continuing to fall. StrongMinds is now treating for ~$35, about a third of GiveWell’s best guess from a few years ago, and is expected to...
The Future of Life Institute Endorses Avoiding the Creation of Digital Minds
In §4 "Human Agency and Liberty" of FLI's Pro-Human AI Declaration, the principle "No AI Personhood" states: "AI systems must not be granted legal personhood, and AI systems should not be designed such that they deserve personhood." (emphasis added)
The second part suggests that we should actively avoid designing sentient AIs. After all, if they were sentient, then they would deserve personhood. To preclude the latter, bar the former.
The likely reason: liability evasion. A companion principle reads: "AI must not be able to act as a liability shield, preventing those deploying it from being legally responsible for their actions." Personhood is an upstream fix: no personhood means there is no entity, other than existing companies, to absorb responsibility.
The contending reason: avoiding power concentration. The broader framing ("Human Agency and Liberty") suggests that AI legal status could concentrate power and dilute human rights protections.
I can't be sure these are FLI's reasons. The conference was closed; deliberations are not public. But all this to say, if these are FLI's reasons, then both reasons are premature.
Both concerns can be addressed by placing AIs within legal structures that make them accountable under existing liability and antitrust frameworks, like A-corps. Using personhood is, at best, a sledgehammer for a nail. At its worst, it's a red herring, and companies are trying to deflect the focus of their responsibility onto prejudices about codifying AI sentience.
A better reason for banning the creation of artificial sentience would be analogous to restricting DNA research, human cloning, or genome editing. In all cases, the process entails parties who cannot represent themselves, competitive pressure if one party takes unilateral action, and irreversibility in cases of large-scale deployment. That said, I think that decision should come from international conversations with international values, and not a bunch of Westerners in New Orleans.
I may be harping; the declaration is not binding. But, I do think it's a bad look. I get that timelines are short, and we're trying to set up the long-term conditions for global flourishing. But if global flourishing is the real goal, then the globe should be involved. That means, minimally, a bit of diversity in the room where it happens. Ideally, it's a transparent process for public accountability.
Let's not forget: we may have no choice in the matter. For all we know, sentience can be an emergent property of current or future architectures. That's not to say §4 is wasted ink, just that it could use a little humility.