Note: I no longer have as much faith in this idea, as I have become increasingly concerned with the threat of populist drift and how little time we may have to improve public educatedness before A(G/S)I, not to mention the relative infeasibility of managing to actually get AI companies to build public input into their models in any meaningful way. I am instead shifting somewhat more toward a holistic approach involving basic human rights-style fundamental values (perhaps involving CEV), input from AI experts and the public, and a safe degree of customisation across geographical and individual contexts. I hope to write a new post on this at some point.

Democratising AI alignment has been put forward as a way of addressing a concerning question facing alignment research thus far: alignment to whom? Whose values are the blueprint for ensuring AI safety? Opening alignment strategy up to public input could offer one approach to issues associated with essentially leaving such crucial decisions up to a handful of tech oligarchs. But while the idea of democratising alignment is compelling, it is fraught with risks. Without careful design, it could lead to poorly aligned AI shaped by populist whims, misinformation, or hasty consensus-building.

Here, I attempt to sketch a framework that acknowledges these tensions and explores a pragmatic, phased approach to democratising alignment.

The Dangers of Oversimplified Democratisation

Efforts to include public input can backfire if not structured carefully:

• Populist Drift: Alignment decisions made via broad, unfiltered public polling risk producing lowest-common-denominator AI behaviour or reinforcing harmful cultural biases.
• Speed vs. Scrutiny: Slow, consultative decision-making could cause alignment-focused labs to fall behind competitors who skip safety checks.
• Fragmentation Risk: Misaligned inputs across different countries or demographics could produce a patchwork of incompatible models or governance rules. This could undermine efforts to create consistent safety norms and open up opportunities for regulatory arbitrage, where bad actors exploit the least restrictive jurisdiction to deploy risky AI systems. Additionally, OpenAI’s grant programme surfaced issues in achieving true diversity across linguistic and digital divides, which risks skewing results in favour of more digitally connected and typically more optimistic participants.
• Polarisation & Misinformation: Like any other political issue, alignment deliberations are vulnerable to ideological echo chambers and fact-free debates. For example, Anthropic’s public constitution experiment revealed distinct value clusters that often diverged strongly on contentious issues, illustrating how polarisation can shape the outcome of alignment processes. Notably, even when trained on public input, the AI exhibited lower bias, suggesting that democratic participation can improve inclusion—but also that navigating strong disagreement is complex and sensitive to group composition and methodology.

At the same time, overcorrecting toward purely technocratic control introduces its own risks: it may overlook ethical and social dimensions, lack legitimacy, and deepen public distrust. A stakeholder-based analysis suggests that because AI affects everyone—directly or indirectly—everyone could have a valid stake in its governance.

A Structured Approach to Democratic Input

Instead of abandoning democratisation due to these risks, we could experiment with institutional and technological systems that help mitigate them:

Layered Participation and Expertise Filters

Democratic input in AI alignment could benefit from balancing openness with structured access. Participation might be grounded in basic AI literacy, potentially via prerequisite courses or orientations in alignment ethics.

Democratisation also needs to grapple with AI’s unusual access profile: models are widely used, increasingly modifiable, and deeply impactful—meaning governance mechanisms could include not only developers, but also affected publics. Multi-phase systems might allow broad citizen input to shape value priorities (e.g., "autonomy over paternalism"), followed by expert interpretation into technical updates—subject to public audit. Expert override mechanisms for safety-critical cases could remain, but perhaps require justification similar to judicial dissent.

Fast Action Under Democratic Constraints

Democratic systems might require mechanisms to support rapid action, particularly in the context of AI. Companies could retain limited emergency powers within pre-agreed value frameworks. An independent, publicly accountable oversight board could retrospectively review such decisions and respond accordingly. This fast-slow hybrid draws on precedents like the War Powers Act and is reflected in ideas from collective governance prototypes.

Misinformation and Polarisation Safeguards

Deliberative systems could be made more robust by introducing a set of safeguards. Public deliberation might be preceded by curated briefing materials vetted by interdisciplinary panels, ideally outside of corporate influence. Platforms such as Pol.is may help identify areas of consensus by clustering viewpoints, limiting amplification of polar extremes. AI tools could assist in surfacing misinformation without censoring it, using flags backed by independent fact-checking sources.

Separating participatory inputs (like large-scale surveys) from structured, smaller-scale deliberations could help maintain clarity, as argued in Lucile Ter-Minassian’s framework. Given that public values shift over time, these systems might be designed to accommodate ongoing engagement. And as Anthropic’s findings show, even structured input produces distinct normative clusters, so consensus shouldn’t be forced where genuine disagreement exists.

Regional Diversity with Global Consistency

AI governance could differentiate between universal terminal values (e.g., safety, dignity) and culturally specific instrumental preferences (e.g., preferences around levels of AI autonomy in scientific research, global coordination, or information access). A Global Core Values Charter might be developed through multinational citizen panels, while more granular behaviours could be guided through co-design tools that let users set their preferences within safety constraints.

Countering Power Asymmetries

Given the imbalance between the public and powerful AI labs, oversight could be strengthened by establishing independent review boards empowered to audit how public input is handled and issue binding recommendations. Escalation mechanisms—for example, enabling a small but significant portion of participants to trigger regulatory review—might support accountability. Since legitimacy stems in part from inclusion, participation models could be designed to rebuild trust in institutions. Incentives—such as those trialled in OpenAI’s grant programme, including payment for underrepresented creative contributions—might help broaden input.

Resilience to Trolling

To protect democratic input from being undermined, systems could combine lightweight identity verification (e.g., proof-of-personhood without breaching anonymity) with “earned trust” reputation systems. Influence might grow with constructive engagement, and be filtered through cluster-mapping rather than raw vote tallies. Moderation frameworks could be transparent and supported by AI to detect and contain bad-faith behaviour, without restricting good-faith disagreement.

What About Dead Internet Bots?

This remains an open challenge. Preventing large-scale manipulation by non-human actors could require governance and technical innovation beyond what currently exists. It may be one of the most critical unresolved risks to meaningful democratic input.

Democratising AI alignment need not mean opening the floodgates uncritically. Rather, it could involve building layered, inclusive, and technically informed processes that allow public input to shape values—without sacrificing safety or coherence. The real risk may lie less in trying to democratise than in doing so carelessly—or not at all.

I invite feedback, criticism, and further suggestions to improve or test this proposed framework.