Most AI-geopolitics analysis concentrates on frontier production. I've been exploring a complementary question: what happens to the capabilities the frontier leaves behind?
Downloadable models can persist after commercial support and meaningful developer control disappear. As inference and adaptation costs decline, some of these systems may become accessible to actors that could not have exploited them at release.
The longer piece develops this through the ideas of capability sedimentation, temporal capability arbitrage and “compute-poor but model-rich” states.
I'm especially interested in whether this should affect how the AI-governance community thinks about model-release decisions and proliferation below the frontier.
Crossposted to Less Wrong
‘Like most of life’s dangers, the dangers of artificial intelligence are not easily identified from the armchair.’
J Dmitri Gallow
Epistemic status
I’m not an expert in this field, and herein I make some sweeping generalisations educated guesses about the overall...
Let’s imagine it’s November 2031. We’re both still here, AI is much more capable than it is today, but governments are governing, civilisation is still (mostly) civilised, and there has been no catastrophic loss of control. It just kind of didn’t happen.
What happens to the AI safety field then?
I expect people who currently believe catastrophe could happen within two or three years will start saying five or ten years. Then 15. Then ‘who cares if we’re a few decades of...
Thanks to Bruce Friedrich, Seth Ariel Green, Tobias Leenaert, Jan Willem Bolderdijk, Martine van Haperen, Herman Rohrbach, Chris Popa and Elizabeth Alvarez for comments. Claude helped structure the text. Ideas and mistakes are my own. I currently work with ProVeg International. The prize I propose would be defined and judged by an independent committee, and its money would fund scaling the winning solution(s), not ProVeg or the winner.
TL/DR
* EA could d...
Most AI-geopolitics analysis concentrates on frontier production. I've been exploring a complementary question: what happens to the capabilities the frontier leaves behind?
Downloadable models can persist after commercial support and meaningful developer control disappear. As inference and adaptation costs decline, some of these systems may become accessible to actors that could not have exploited them at release.
The longer piece develops this through the ideas of capability sedimentation, temporal capability arbitrage and “compute-poor but model-rich” states.
I'm especially interested in whether this should affect how the AI-governance community thinks about model-release decisions and proliferation below the frontier.
https://fabiofrettoli.substack.com/p/the-geopolitics-of-model-graveyards?r=sb1v&utm_campaign=post-expanded-share&utm_medium=web