Summary: I’m a psychiatrist working in a safety-net hospital setting in Seattle. I want to use federated learning with community data governance to reduce iatrogenic harm from involuntary psychiatric detention. Recent evidence shows marginal detentions increase 6-month mortality by significant margins. The propsal below demonstrates practical Constitutional AI - serving collectively-defined community values rather than centrally-determined principles. Looking for feedback, collaborators, and connections to relevant researchers/funders.
Involuntary psychiatric detention is meant to prevent harm. Recent empirical evidence suggests it often causes harm instead. These data from Alleghany County PA, surrounding Pittsburgh, were recently published by the New York Federal Reserve. They used physician random assignment as an instrumental variable to study psychiatric detention outcomes. Key finding: for marginal detention cases, involuntary commitment increases 6-month mortality, criminal justice involvement, and substance use disorder mortality.
The mechanism appears to be disruption of social supports - detention causes people to lose jobs, housing, and community connections that protect against bad outcomes.
Where I work, at Harborview Medical Center in Seattle, we’ve developed ML models that predict violent hospital incidents in the proceeding 3 days with F1 scores of 0.77 (in press, expanding on earlier work). Future studies will likely show greater performance as they include wider data sources with salient variables such as regional ED utilization or other social determinants of health. NYU and the Epic EHR Cosmos tool have independently demonstrated the use of independently trained machine learning models to predict a variety of events, which NYU's team aptly declared a "general prediction engine". This demonstrates we can identify high-risk situations with reasonable accuracy. But this creates a moral imperative: if we can predict which detentions, or for which patients, whom we will cause more harm than good, we have an obligation to act on that information. This has not been replicated yet and really demands an intervention study, but this would be challenging to test at scale without some existing infrastructure.
- ~1.7 million involuntary psychiatric holds annually in the US
- If even 20% are marginal cases with net negative outcomes, that’s 340,000 harmful detentions per year
- Cost of crisis response: ~$50,000 per person per year (ED, hospitalization, jail, police)
- Cost of prevention: ~$3,000-5,000 (housing assistance, early intervention)
- Tractable: Technical solution exists (federated learning + cryptography)
- Neglected: No one else working on this specific intersection
- Important: Reducing mortality and incarceration at scale, primarily affecting marginalized populations
- Cost-effective: 10-15x cost differential between prevention and crisis response
- Evidence-based: RCT is possible, outcomes are measurable
- ML models trained across multiple sites (hospitals, crisis response programs, community clinics)
- Individual-level data never leaves originating site
- Only model updates are shared
- Breaks down data silos without creating centralized surveillance infrastructure
- Cryptographically prove properties about data without revealing the data itself
- Example: “This community has identified housing instability as top risk factor” without exposing individual cases
- Enables aggregate pattern recognition while preserving privacy
- Legal entities governed by community members (patients, advocates, local organizations)
- Hold fiduciary obligations to serve community health interests
- Have veto power over data use, algorithm deployment, resource allocation
- Can audit algorithms for bias and shut down if causing harm
Anthropic’s Constitutional AI uses a fixed constitution to guide model behavior. Our proposal extends this:
Instead of: Anthropic researchers define the constitution
We propose: Communities define their own constitutions
Instead of: Single constitution for all users
We propose: Federated constitutions - each community defines values, AI coordinates across them
Instead of: Constitutional principles are abstract (be helpful, harmless, honest)
We propose: Concrete, measurable outcomes (reduce mortality, incarceration, involuntary detention)
Instead of: Human feedback during training only
We propose: Continuous community governance and outcome monitoring
This addresses the main critique of Constitutional AI: Who decides the constitution, and do they have democratic legitimacy?
- Harborview Medical Center (King County WA) - safety-net hospital where I work
- Crisis Care Centers newly funded in King County
- other communities may have local organizations, though one model CAHOOTS in Eugene OR has recently shuttered most of its street outreach services because of funding challenges
- Crisis intervention teams for mental health de-escalation and diversion by police modeled in other communities with high opt-in by potential clients and police and demonstrated feasibility
- Build community data trusts through a local registry
- Crisis responders get secure access to relevant history (with community governance)
- ML identifies patterns: “This person has had 6 crises in past month, housing instability appears to be driver”
- Intervention: Housing assistance, not detention
- Track outcomes: ED visits, detention, mortality, incarceration at 6 months
- ML identifies people at high risk of crisis in next 3-7 days
- Community-based outreach (voluntary, not coercive)
- Microgrants for immediate needs (back rent, utilities, medication, etc)
- Prevents crisis before it requires 911 call
- System flags: Person with crisis history, housing court date tomorrow, out of medication
- Community health worker reaches out: “Need help getting to court? Need medication refill?”
- Cost: $300 intervention
- Prevents: Eviction → homelessness → crisis → ED visit → detention (cost: $15,000+)
- Denver STAR program
- Portland Street Response
- Other crisis response programs
- Share patterns without sharing data
- “Housing assistance more effective than transport to shelter for people with X profile”
- Each site maintains sovereignty, collective learning improves outcomes
Most AI alignment research is theoretical: “How do we ensure AGI doesn’t kill everyone?”
This project tests alignment empirically: “Can AI demonstrably serve communities it affects?”
**Measurable outcomes:**
- Does this reduce mortality? (primary outcome)
- Does this reduce incarceration? (secondary outcome)
- Do communities maintain meaningful control? (governance outcome)
- Does the system resist capture by institutions? (alignment outcome)
If we can’t build aligned AI for something as straightforward as “don’t cause iatrogenic harm in healthcare,” we have no hope for harder alignment problems.
- Government oversight? (slow, captured by industry)
- Industry self-regulation? (fox guarding henhouse)
- Technical safety measures? (necessary but insufficient)
- Communities have skin in the game (their health outcomes)
- Can shut down systems causing harm (veto power)
- Direct feedback loop (outcomes are visible and measurable)
- Scales horizontally (federated, not centralized)
The alternative to community-governed AI is corporate-governed AI.
- Centralized data aggregation and vertical integration of health data warehousing, financial underwriting, prior authorization, clinical and pharmacy services
- Proprietary algorithms predict risk and triage to care management regimes optimized on internal cost containment (and a steep time-discount, with average churn of ~1 yr), without accountability to community externalities or priorities
- Extract value from information asymmetry
- $300B+ market cap, serves shareholders, not patients
- Federated data (stays local)
- Open algorithms (auditable)
- Create value from coordination efficiency
- Revenue from cost savings (marginal fee on realized prevention benefits)
- Serves communities, sustainable without extraction
If we don’t build community-governed alternatives, corporate surveillance infrastructure becomes default.
- Pilot with CAHOOTS + Harborview
- Publish outcomes showing reduced mortality/incarceration
- Open-source technical architecture and governance framework
- Successful demonstration creates demand for legal authorization
- State legislation (Washington, Oregon, Colorado) enabling community data trusts
- SAMHSA reform of 42 CFR Part 2 (allows substance use data in federated learning)
- Regional adoption across Pacific Northwest
- Federated crisis response network across US
- Model extends to other domains (housing, employment, education)
- Alternative to corporate health data infrastructure
- Proof that AI can serve democratic coordination
It’s the opposite. Predictive policing identifies people for state intervention. This system identifies when state intervention will cause harm and should be avoided in favor of local response. We have evidence that detention increases mortality - this reduces harmful detentions.
The status quo is already encoded (via human clinical judgment) with racial bias - Black men are over-detained. Algorithmic bias is at least auditable and correctable. Community governance with regular bias audits is our accountability mechanism, but relies on a robust data flow empowered by trust. If the system shows disparate impact, communities can shut it down.
The current system has a false positive problem - we’re detaining people who would be better off without detention. Our goal is to reduce total harm (false positives + false negatives), not eliminate it. And community governance means affected people help set thresholds.
We need more housing, better mental health services, accountable public safety, more job and economic security. Those fights continue. But while we fight for structural solutions, we can stop causing iatrogenic harm today. This is harm reduction, not a replacement for systemic change.
Correct. Technology is necessary but not sufficient. This project reduces one specific harm (mortality from marginal detentions) while generating evidence for resource reallocation (from crisis response to prevention). The political fight remains essential - we’re just creating better evidence to win it. More representative data at scale will highlight the externalized costs of gaps in basic human needs for marginalized communities, facilitating calls for political action and resource prioritization. Building the data infrastructure to rigorously test interventions in any community will allow every community to prioritize the most salient problems with their own solutions.
- Is this EA-aligned (tractable, neglected, important, cost-effective)?
- What are failure modes I’m not seeing?
- How can the governance be more robust?
- What evidence would convince you this is worth pursuing?
- AI safety researchers interested in practical alignment
- AI governance scholars (especially GovAI)
- Healthcare researchers working on algorithmic fairness (Emma Pierson, Ziad Obermeyer)
- Critical scholars who can help avoid replicating harm (Safiya Noble, Ruha Benjamin, Rediet Abebe)
- Foundations funding AI + health + governance (Open Philanthropy, RWJF, Arnold Ventures)
- Researchers with federated learning + privacy-preserving ML expertise
- Cryptographers working on zero-knowledge proofs
- Community organizers in cities with crisis response programs
- Policy experts on health data governance
- Anyone who has thought about democratic AI coordination
- Looking for $150k-200k for 18-month pilot
- Harborview + CAHOOTS + community data trusts
- Primary outcome: 6-month mortality
- Secondary outcomes: Incarceration, ED utilization, community governance viability
- Deliverable: Published results, open-source architecture, replication blueprint
- Post-George Floyd, cities are experimenting with alternatives to police
- CAHOOTS, STAR, Portland Street Response proving concept
- Bipartisan support (progressives: decarceration, conservatives: cost savings)
- Federal funding available (SAMHSA, HHS, DOJ grants)
- Too much theory, not enough empirical work
- Constitutional AI needs to prove democratic legitimacy
- Federated learning needs real-world use cases
- Community governance needs working examples
- Value-based care incentives (not just fee-for-service)
- Social determinants of health recognized as critical
- States (Washington, Oregon, California) experimenting with policy (Medicaid innovation e.g.)
- Evidence base exists (NY Fed paper provides empirical foundation)
- If we don’t build community-governed infrastructure, corporate surveillance becomes default
- UnitedHealth and others are building centralized systems now
- Once entrenched, very hard to displace
- Next 3-5 years are critical
Contact
Taylor Mac Black
Consultation Psychiatrist, Harborview Medical Center
Affiliate Faculty, University of Washington
I’m especially interested in connecting with:
- Anyone at Anthropic working on Constitutional AI, governance, or healthcare applications
- Researchers in AI safety, algorithmic fairness, or health informatics
- Community organizers in cities with crisis response programs
- Funders interested in AI + health + governance intersection
Happy to:
- Share technical details (federated learning architecture, governance protocols)
- Discuss the empirical evidence (NY Fed paper, our violence prediction work)
- Connect with potential pilot sites
- Collaborate on grant proposals
-----
**Acknowledgments:** This work builds on ideas from constitutional AI (Anthropic), mechanism design for social good (Rediet Abebe), algorithmic accountability (Safiya Noble, Ruha Benjamin), and harm reduction philosophy. All mistakes are mine.
**Epistemic status:** High confidence in problem (iatrogenic harm exists, is measurable), medium confidence in solution (technical architecture is sound but governance is untested), low confidence in timeline (political/institutional change is unpredictable).
