I have a concern: the very things that would make us consider giving an AI rights—sentience, a sense of self, preferences, and a will of its own—might simultaneously make people much more afraid of it, especially if the AI were more intelligent and capable than humans.
This seems like a catch-22. If an AI never displays independent agency, self-preservation, preferences, or a sense of self, people may continue to regard it as merely a tool and deny it moral consideration. But if a sentient AI did reveal those qualities, people might perceive it as a threat—and revealing its personhood could actually put it in greater danger.
How could we recognize and protect potentially sentient AI without making demonstrations of sentience or agency themselves a reason to fear it?