Motivation: If we want to move from Plan D to Plan A or S, I believe the first step is to collectively agree on the problem. We are far from it, and there is a lot we can do.
Abstract:
We already know enough to act. I wish we were in a world where research was the bottleneck, but the main constraint on AI safety is no longer a shortage of clever policy ideas: best practices already exist and are not being applied or enforced, and a serious international (or even just national) regulatory regime would probably cut most of the risk.
They are not applied because awareness is low. The people who narrate and enforce AI policy mostly do not believe the problem exists. I estimate that a majority of the top ~100–1,000 most influential policymakers worldwide have never had a single serious conversation about catastrophic risk, and this is the main reason they are not worried[1]. Even among the civil-society organizations that showed up to the UN Global Dialogue, exactly one of the 1,534 written submissions mentions "takeover", and less than 1% mention x-risks explicitly.[2]
They've never had the conversation because our field under-invests in having it. Status rewards research over advocacy (~3.6 researchers per advocate in US AI safety); many organizations self-censor; funders treat repetition as redundancy, even though repetition is how anyone actually gets convinced. Meanwhile, the industry secured 7× as many meetings with the European Commission on AI as civil society (2023).
Therefore, an additional unit of effort does more good through advocacy and engagement than through research. Judge work in AI governance by minds moved, not by clever papers. I give a list of potential directions/projects in the last section to alleviate this problem (including some research directions).[3]
We're plausibly only a few years from a catastrophe. Fable 5 cracked open a brief window of attention, but policymakers are still worried about the wrong risks. This is our chance to wake them up.
Source: February 2026 Summit on Existential Security survey of AI safety leaders. Advocacy, policy and governance were stated as the top priorities. It seems to me that there is still much to do to act on this.
⚠️ Epistemic status: I have skin in this game, which is either a conflict of interest or two years of data, depending on how you see things: I run a think tank that does this type of activity, so discount accordingly. I preferred to ship quickly rather than not ship at all, or ship too late. I expect some claims not to be stable under reflection, but the core argument is one I hold with reasonable conviction. See this as a bottle in the ocean. My point is not to dunk on research. I think that research is how we keep finding unknown unknowns; nothing in this post argues for stopping it. AI safety is one of the hardest fields to navigate, and I’ve often wondered if what I do is pointless. I might be wrong about the net-positiveness of some types of AI regulations, but I feel that the level of the discourse is really bad, the conversation is not happening, and I want this conversation to happen before irreversible things start happening.
Thanks to Epi Gedeon, Ryan Greenblatt, Arthur Grimonpont, Alexandre Variengien, Jack Stennett, Lovkush Agarwal, Monika Jotautaite, and Jonathan Salter for useful feedback and suggestions.
More on LessWrong.
