Tom Milton, CEO of Amodo Design — which builds hardware for scientific research, biosecurity, and AI verification — argues that AI chips are a great place to start checking whether rival AI developers are really slowing down, and that the technology to do it is “tractable but not solved.”
How can you verify a data centre is only running inference?
Amodo’s pilot uses inference recomputation: a network tap copies a cluster’s front-end traffic to the verifier’s own server, which reruns the agreed model to check the outputs match, and any traffic it can’t explain points to an undeclared workload. On Tom’s account, the pilot runs on about 30 to 32 H100 and H200 GPUs. Because the verifier currently needs the model weights, he expects real deals to rely on zero-knowledge proofs instead; these carry many orders of magnitude more overhead, but checking one packet in 1,000 could give reasonable confidence.
Chemical weapons inspections offer a model for AI verification
Inspectors from the Organisation for the Prohibition of Chemical Weapons (OPCW) give 48 hours’ notice before a standard inspection of a declared site, and use mass spectrometers with a confidential mode that returns only a pass or fail. AI verification needs similar confidentiality to protect model weights and users’ input and output tokens, Tom argues. The effort gap is large, on his estimate: maybe only 50 engineers in the world work on compute verification, while the US spent billions of dollars in today’s money on Project Vela to detect nuclear tests.
A real deal might be put together in two to 20 days, Tom expects. If a deal happened next month, he says, zero hardware verification mechanisms would be used; the fallback would probably be shutting down data centres, sending in auditors, and using infrared satellites. Long-lead-time work should start now, including Amodo’s network certifier, an FPGA chip that keeps a cryptographic log of network traffic and still needs security testing, red-teaming, and scaled-up production.
Why AI lab researchers should work on compute verification
What would most help Tom sleep at night is dozen-person verification teams at the major AI developers. Given how often lab staff cite verification in their theory of victory, an outsider would expect 10 multihundred-person teams, but he suspects the real effort is much smaller. In his view, a talented safety or security researcher who started today would probably be one of the best five people in the world working on verification by the end of the year, because lab staff know the frontier and infrastructure, and are who leaders will ask when a deal is on the table.
How good are AI models at designing circuit boards?
Claude Fable is the first model that could contribute to hardware design at all, Tom says, roughly where GPT-3.5 or GPT-4 were for software. A power-monitoring circuit board that took an Amodo engineer 10 to 30 hours to design, Fable produced in one shot; it works, but is extremely ugly and hard to debug. He puts Fable 5’s hardware time horizon at something between 10 and 100 hours, and finds models OK at schematics, poor at layout, and very bad at routing.
This summary was drafted with AI from the episode transcript and reviewed by the 80,000 Hours team.
