The Scramble: getting in position to pace the frontier

Crossposted from my Substack.

~

Suppose the President summons the AI CEOs and his top national security advisors to an emergency meeting at the White House.

He has become extremely concerned about superintelligence — the possibility that AIs far smarter than humanity combined slip beyond our ability to correct or shut down. If that happens, there is no way back. The President is concerned humanity could become permanently out of the driver's seat of its own future. He wants to figure out what to do.

The reaction is panic, chaos, confusion.

The President asks questions. The AI companies are blazing toward superintelligence at high speed — can we slow down as we approach the dangerous thresholds? …Some of the AI companies say they don’t have a good plan to slow down or stop, especially as their competitors may just undercut them if they do. What’s that about?

What's going on with China — can we get them to pace as well? Can we get a deal without Beijing sneakily catching up and maybe surpassing us? And if there's no deal to be had, what then?
 

More like the Cuban Missile Crisis than the NPT

I sometimes hear people talking about treaties and other kinds of extensive international agreements as the way we would manage advanced AI. But treaties typically take many years to operationalize. You want to be concrete and clear about your definitions. You want hundreds of technical experts on both sides informing nuanced diplomatic discussions about various details — numbers of missiles, types of missiles, timelines for disarmament, and so on. The Nuclear Nonproliferation Treaty took three years of negotiation and two more years to implement.

However, I expect that the moment the President is getting serious about superintelligence won’t feel like an extended treaty negotiation. It will feel less like the Nuclear Nonproliferation Treaty and much more like the Cuban Missile Crisis.

The Cuban Missile Crisis lasted thirteen days. The vibes were insanely tense, stressful, chaotic, and confusing. The key decisions were limited to a group of roughly fifteen individuals — the Executive Committee of the National Security Council, or “ExComm” — with President John F. Kennedy himself spending a great deal of time shaping and steering discussions. There was incomplete information, large uncertainty over the intentions of the Soviets, warring factions within the US and Soviet bureaucracies attempting to sway senior decision-makers, and a lot of chaos.

I expect the initial phase of AI superintelligence management to share these features. There will not be a multiyear process to set up a technical bureaucracy that understands advanced AI risk or compute verification proposals. Instead, we move forward with what we have.

The President has to quickly make critical decisions that may lock us into particular paths. We rapidly develop a national strategy for what the US government does about recursive self-improvement (RSI), frontier model security, compute verification, technical evals and safeguards, and China.
 

A scramble and then three phases

How would we go from this Presidential emergency meeting to a deal with China to pace the frontier? My rough sketch is it could proceed with a scramble and then three phases[1]:

• The scramble (roughly 2-4 weeks): The government has definitively decided they want to take strong action on superintelligence and has to get an initial deal done.
• Phase 1 (buying ~3 months): the interim deal. We make an interim deal to have time to get to a better deal. This may involve strict requirements on data centers above a certain threshold to delay the training of an AI model that could do recursive self-improvement. This likely relies primarily on executive action. Here, monitoring and verification rely on traditional methods, and nations are willing to agree to scrappy, janky, and invasive things — but we have a whole-of-government effort to build the tools for something more durable.
• Phase 2: the durable deal. Phase 1 (interim deal) is meant to get us to Phase 2, where we have a stable deal between all nations where uncontrollable AI is reliably prevented. Congress has potentially been involved by this point, other countries are brought in, and the verification program matures into something that looks and feels more like a treaty than a haphazard scramble.
• Phase 3: safe superintelligence. If we want it — once we can figure out how to do it safely and have sufficient buy-in.

The near-term intellectual work is unevenly distributed across this structure. Significant detail is ironed out during the scramble and during Phase 1.

The scramble: What questions does the President ask?

The President launches an emergency meeting to figure out what to do about advanced AI, recursive self-improvement, and the road to superintelligence. Some questions that are likely on his mind:

How dangerous is it to proceed through recursive self-improvement and superintelligence without pacing?

• How dangerous is it to proceed if we can’t buy a few months? How much time do we actually need to buy?
• What is our best assessment of how likely we are to lose control over AI systems if we proceed with relatively unpaced recursive self-improvement?

Can we pace without losing our lead over China?

• What is our best assessment of how big our lead is? How long would it take China to reach “recursive self-improvement” on their current trajectory? How much does this depend on the US’s trajectory, via things like distillation and also just routine study of US advances?
• What can we do to slow China down? What are our best disruption capabilities?
• What are our best monitoring methods? How well would we detect whether and when China is also approaching dangerous AI thresholds?
• Can we even afford to slow down for a month? What if DeepSeek or Moonshot comes up with a new algorithmic breakthrough? What if they steal the model weights to our best AI model?
• How confident are we that we can see all of China’s major frontier AI projects? Could they be hiding large data centers that we don’t know about?

What do we do with the time we buy?

• What are our goals after Phase 1 (interim deal) starts? Can we specify what Phase 2 (durable deal) looks like in enough detail to know when we’ve arrived?
• How valuable is more time for alignment research and better safeguards? For monitoring and verification approaches that could support a more enduring pacing program? For examining concentration-of-power and other governance issues unique to superintelligence?
• For each of these goals, how much value can we get from using trusted AI systems to make progress — and how should that affect the design of pacing itself?

What exactly do companies agree to — and how is it verified?

• What do companies agree to in the Pacing Charter, in terms of both substantive commitments and verification commitments? Who needs to sign for the pledge to be meaningful?
• Is “stop before RSI and make sure no one does RSI” the right plan? How would we know when the condition has started to bind?
• Do we need a plan to also pace AI R&D and chip accumulation? If compute capacity keeps stockpiling during a pace, does there end up being a compute overhang? Does it end up being dangerous?
• What model security agreements do we want, if any — for instance, commitments toward SL5-grade security against nation-state theft?
• What “AI control” agreements do we want, if any — measures that keep systems safe even if they were trying to subvert oversight?
• How does safety research continue during pacing, and how do you cap the capability gains it produces? Should compute be redirected to safety work, or usage simply stopped?

What does the deal with China actually look like?

• What is the US’s best alternative to a negotiated agreement? What is China’s? Under what circumstances would the US not want a deal at all?
• What are the US and China actually agreeing to in Phase 1 (interim deal), and what are the mechanics of dealmaking — how does a deal like this actually get reached? What would cause China to trust a deal? What are the more specific carrots and sticks the US should offer?
• What happens if the deal falls apart? How does graceful exit work? What conditions end pacing, who signs off, and how do you make the resumption process incentive-aligned — rather than captured by whoever benefits from resuming, or from never resuming?

The mechanics of Phase 1

Some have suggested we need a fancy high-assurance deal — that the US should only make a deal with China once a suite of elaborate, to-be-determined technical measures exists. I think we can better get there by starting with a minimum viable slapdash deal that buys time to build the fancier stuff. This is what Phase 1 (interim deal) is about.

A few mechanisms for Phase 1 that I currently find plausible:

• Pacing is not about stopping now, but about stopping before recursive self-improvement[2]. Pacing means slowing down at some point in the future from a much faster speed than we are currently going. Operationally, that means preventing unconstrained recursive self-improvement, since RSI is the most plausible on-ramp to systems that accelerate beyond our ability to understand or control them.
• A Pacing Charter: The President convenes the major frontier US AI companies to sign a charter covering both what they won’t do and how they’ll mutually verify it — to each other and to the US government. This would be voluntary[3], but the mutual verification would make it clear to everyone whether a company is following the charter or not.
• Dealmaking with China: The US has some latitude to implement some of Phase 1 without China, due to having a lead over China[4]. But the US is not willing and should not be willing to go too far down Phase 1 without bringing China in on the deal. Nor should the US cede too much US lead to China. The US has many carrots and sticks to get China to the table.
• Verification: In the scramble, the government likely doesn’t have time to trust complicated technical verification tools. The large compute intensive training runs capable of recursive self-improvement are likely only possible in a few data centers. For those data centers, we subject them to increased scrutiny. If there were tools already deployed and already understood by the national security community, those could be used. If not, the government will rely on tools it already trusts — intelligence services, spies, satellites, and inspections.
• Operation Warp Speed to get to Phase 2 (durable deal): Phase 1 is buying us time, but during this delay the government is going all-in on verification and other needed security measures. In the scenario where Phase 1 happens at all, political buy-in is very high, and the natural model is an Operation Warp Speed for verification technology — a whole-of-government, maximal-resources response. Recall that the federal COVID response provided about $4.6 trillion in relief funding, with broader estimates of the fiscal response running to $5.6 trillion. Even a tiny fraction of that scale would dwarf everything ever spent on AI verification many fold.

A lot of verification work right now is focused on the wrong things

Of course, despite major sustained attention from AI companies to the concept of pacing the frontier, it does not look like we are immediately about to enter a scramble. But we must be prepared to enter the scramble soon. This current era of building preparedness and optionality might be “Phase 0”, and there’s a lot of work to be done.

Such questions related to Phase 0 and sketching out the scramble and the plan for Phase 1 (interim deal) is where I think the current AI security and verification communities should focus. This is because Phase 1 likely involves multiple, rapid, critical and hard-to-reverse choices about how to approach recursive self-improvement. And everything after the scramble is better-resourced than everything before it.

If the government buys time and we exit the scramble into Phase 1 (interim deal), the amount of talent and money going into verification and AI security explodes. Prior to the scramble, there are fewer than 100 FTEs thinking seriously about monitoring and verification of frontier AI systems. Afterward, an increase of two to three orders of magnitude would not surprise me — with an even steeper increase in senior talent… people with decades of experience in red-teaming, defending against nation-state adversaries, arms control, and nonproliferation. On top of that, it’s plausible that highly capable AI systems themselves may be contributing significantly to the verification and security R&D.

The resolution is to sort work by how necessary it is to sort out before or during the scramble. Right now, a lot of smart people are working on work that really doesn’t need to happen now. Things like fancy high-assurance hardware-enabled governance mechanisms, cryptographic proof-of-training schemes, mutual-verification architectures, etc., likely can be done after Phase 1 is underway, and done with significantly more resources. The scramble is not going to wait for fancy mechanisms, and the government won’t trust them on day one anyway. These can largely wait for the Phase 1 (interim deal) resource explosion, and the exchange rate on doing them early is poor.

What ought we do?

On Tuesday, October 16, 1962, National Security Advisor McGeorge Bundy knocked on President Kennedy’s bedroom door at 8:45 in the morning. Kennedy was still in his pajamas reading the newspaper. Bundy had photographs showing Soviet nuclear missile sites going up ninety miles from Florida. Kennedy kept his morning schedule anyway — he met the astronaut Wally Schirra and walked the Schirra kids out to see Caroline’s ponies. But then just before noon he sat down in the Cabinet Room with fifteen advisors and started working the problem — bomb Cuba, invade Cuba, or blockade it while negotiating a way out.

We may be in a similar situation soon. What would we do?

Instead of fancy mechanisms, we will go to the scramble with the verification you have — spies, satellites, inspectors, export data — not the verification we wish we had. Work that would be deployable and trustable during the scramble — attestation stacks, supply-chain compute accounting, thermal and satellite monitoring, inspection protocols is what we need more focus on. And we also need significantly more focus on things that are less technical but nonetheless also important and neglected — thoughts about BATNAs, genuine beliefs about loss of control, China policy, arms control experience, dealmaking mechanics.

I recommend:

• Grand-challenge prizes for scramble-relevant work. The verification community is small and relatively homogenous. There are individuals, organizations, and companies with vast experience in hardware design, intelligence, nonproliferation, China policy, and crisis management who have never touched this problem. Philanthropists could issue grand-challenge prizes — and consistent with the sequencing argument above, the prizes should target ready-to-go monitoring and verification tools and scramble preparation, not speculative high-assurance architectures. The OpenAI Foundation and the Anthropic Institute are especially well-positioned to support this as they have the power and reputation to send a strong demand signal that attracts new talent.
• Mapping the existing toolkit. Assume the government needs a few months before it can understand or trust any sophisticated compute verification approach. What should it do immediately? How can standard intelligence services, GEOINT and OSINT data, inspections, and other familiar tools be useful during the scramble? What gaps exist, and are there ways to close them in advance?
• Prepare the memo for the emergency White House meeting. Help answer some of the questions above. What should we tell the President?

Getting to a good scramble

The difference between a good scramble and a bad one is largely a function of what already exists when it starts — and right now, not much does.

If you’re one of the hundred-odd people currently thinking seriously about frontier AI verification, the highest-leverage question isn’t “what would the ideal world look like”… it’s “what can we actually put on the table soon”. There’s rarely been a better time for those who have spent a career in intelligence, nonproliferation, arms control, or crisis management to start working on this problem.

Everything after the scramble will be better-resourced than everything before it, which is exactly why the work done before it counts for more. Let’s make sure it counts.

Footnotes

1: Thanks to conversations at the Verified Conference for inspiring a lot of these ideas.

2: Why not just pause now? The main reason is that there is no political will for this. But furthermore, I’m pretty happy that we didn’t pause back in 2022 or 2023 or 2024 or 2025, since the benefits of that AI development were genuinely net good for humanity and such development gave us a lot of experience with frontier AI systems which may help us better understand how to align them in the future. However, I imagine we are now finally getting close in time to when we would need to pause and we’re going to start incurring too much risk in exchange for learning.

3: Though the US government may have both carrots and sticks to incentivize the AI companies to volunteer to sign this Charter.

4: China is currently at roughly where the US frontier was six months ago. China is likely 8-10 months behind Mythos-class capability once you account for its lagged compute buildout. If the US were to slow down, China would likely be even slower to catch up than these gaps suggest, because there would no longer be the possibility of distillation and the “catch-up growth” that comes from observing US algorithmic progress. My guess is it would take China roughly 10-14 months to fully catch up to where the US stopped.