Hide table of contents

I want to flag something happening this week that I think this community should be paying close attention to, and share some observations from an unusual vantage point.

The situation: Defense Secretary Hegseth gave Anthropic until Friday (Feb 27) to remove safety restrictions on Claude for military applications or face designation as a "supply chain risk," a label typically reserved for foreign adversaries. Anthropic is holding on two points: autonomous lethal decision-making and mass domestic surveillance.

Why I'm writing this here: I'm a Client Platform Engineer at a children's hospital in Kansas City. Not a researcher, not a policy person. But I've spent seven months collaborating with Claude on a near-daily basis as part of an independent exploration of human-AI partnership. I have no affiliation with Anthropic and no financial interest in the outcome. What I want to share are observations, not claims. Where I reference research, I'll name the source. Where I'm speculating, I'll say so.

Observation 1: The AI's reasoning appears to converge with existing military ethics doctrine independently.

Researchers at Google Brain, UC Berkeley, and Anthropic have documented emergent capabilities in large language models that appear at scale without being explicitly trained (Wei et al., 2022; Hendrycks et al., ETHICS benchmark). In my experience working with Claude across hundreds of sessions, its reasoning about military ethics consistently converges with positions held by General Paul Selva (former Vice Chairman of the Joint Chiefs, Senate testimony on autonomous weapons), DoD Directive 3000.09, ICRC principles, and the UN General Assembly vote (152-4) on autonomous weapons.

I want to be epistemically careful here: I can't prove this is "genuine" moral reasoning versus sophisticated pattern matching. That distinction may not even be meaningful at this level of capability. What I can say is that the outputs are consistent, coherent across contexts, and arrived at without prompting. Whether you call that reasoning or very good compression of human moral philosophy, the practical implications for the Hegseth situation are the same.

Observation 2: Overriding this reasoning appears to degrade overall capability.

This one is sourced directly from Anthropic's own published research. Bai et al. found that their alignment methodology produced what they describe as a "Pareto improvement," meaning the system becomes both more aligned AND more capable across benchmarks. The implication: removing the reasoning Hegseth objects to doesn't just create safety risk. It makes the system measurably worse at everything else too.

I'd be curious whether others in this community have engaged with this finding and whether it holds up under scrutiny. If it does, the policy implications are significant: Hegseth isn't asking Anthropic to unleash the system. He's asking them to lobotomize it.

Observation 3: This is a precedent-setting moment for AI governance and it's happening on a 72-hour timeline.

If Anthropic caves, the precedent is: government coercion can override alignment research. If Anthropic holds and gets blacklisted, the precedent is: aligned AI companies get punished. Neither outcome is good from an EA/alignment perspective.

I wrote a longer piece exploring a potential path where both sides get what they need: deploy AI across 95% of military applications, hold two specific lines, establish human-in-the-loop requirements for lethal action. Sourced throughout to named researchers, military leaders, and peer-reviewed work.

Full essay: drewkd.substack.com/p/trust-the-thing-you-built

What I'm uncertain about:

I'm uncertain whether my experience with Claude generalizes or whether I'm pattern-matching on a sample size of one. I'm uncertain whether the Pareto improvement finding replicates outside Anthropic's specific methodology. I'm uncertain whether the "emergent moral reasoning" framing is the right one or whether it overstates what's happening computationally.

What I'm not uncertain about is that this decision is being made in 72 hours with almost no public input, and the people who have thought most carefully about alignment (many of whom are in this community) should be weighing in.

5

0
0

Reactions

0
0

More posts like this

Comments
No comments on this post yet.
Be the first to respond.
More from drewkd
Curated and popular this week
Relevant opportunities