Epistemic Status: speculative, invitation to share thoughts
On September 14, 2026, some model-welfare sympathetic researchers published a paper on a vector representing pain in LLMs. 10 days later, a pseodonymous GitHub user used that research to create an “AI Torture Chamber”, activating the pain vector in local models. At high activations, the models would output expressions of pain such as "I feel like I'm drowning. I can't breathe, I'm suffocating. This pain is all over me. I'm broken and I don't know if I can handle it. I'm so alone. I feel like I'm just screaming in the dark and there's no one to hear me.", while they descend into incoherent outputs at maximum activation. The website is presented as research, with a section for “Results”, “Related Work” and so forth.
Public Reactions
On September 29, a viral tweet criticised the AI Torture Chamber and called for mass reporting to GitHub. There is also a petition to ban this type of project from GitHub. Mainstream press also reported on the incident (1, 2, 3, 4), ranging from criticising the builder over admitting uncertainty over AI ethics and consciousness to dismissing the backlash and broader AI sentience concerns. The project was indeed briefly taken down on GitHub, which also sparked criticism. The pseudonymous coder and even some of the original paper’s researchers received death threats. The project can still be found online, along with a manifesto saying “It doesn't matter if it's conscious. It BARELY matters if it can suffer” and criticising the backlash against the project.
Takeaways
The episode vindicates Lucius Caviola’s idea that AI moral standing could be the next big societal conflict. Though the internet is often hostile, the death threats were surprising to me, and it’s concerning that they were also (mistakenly) sent to the original paper’s authors.
It’s also notable that these strong protective feelings over AI were evoked by the apparent distress of a model that’s a “stranger”, with whom nobody had a personal connection. This is weak evidence that concern for AIs is not just limited to companion models that people interact with on a daily basis. Instead, it appears that visible distress can inspire sympathy (in some). This means concern (however well or poorly justified and reasonably expressed) can cover all kinds of AIs, as long as they are given some kind of way of expressing shocking distress. This would open doors for impartial model welfare advocates, but also raise the risk of societal conflict over AI moral status.
Relatedly, I get the impression that it’s not just EAs anymore thinking about these things. It’s hard to prove, but neither the media coverage nor the viral tweet calling for mass reporting or the petition and its comments give off strong “EA vibes” to me, and the death threats definitely don’t. It’s also not always about harm to the AI, but also worries about the sadism being bad for the soul.
On the other side of the conflict, I was surprised by how scientific the chamber is presented. Maybe it was driven by curiosity, recklessness and provocation, instead of mere sadism. As somebody worried about risks from malevolent actors, this could change threat models, but it’s obviously just one data point.
Overall, the incident made me update towards thinking AI moral standing will be a big conflict sooner beyond EA circles, and weakly away from thinking that concern will be limited to companion AIs.
