This is the summary of a longer essay. Full essay: https://beatthedynamics.org
AI safety is not only technical. It is geopolitical: how do rival powers stop a race none of them can afford to lose?
This essay explores the dynamics driving AI development, national AI strategies and AI risk, as well as counter-mechanisms to these dynamics. It does not assume good will. It takes geopolitics as it is, often cold and ruthless, and still points to hopeful solutions. The case it makes is that there is a robust path for nations to coordinate on AI development and AI risk mitigation that would allow us to reduce the catastrophic risks as close to zero as possible and at the same time provide a path to greater and shared prosperity for all nations. Furthermore, it highlights how an AI race as it is currently being pursued would lead to a highly volatile world in which dangers from AI and from geopolitical conflict would become very hard to control and mitigate. Lastly, it shows what a framework on the safer path could look like, how it would be geopolitically realistic, and what we need to concretely do to build it and steer away from the more volatile and dangerous path. It is written for everyone, assuming no prior knowledge of AI or geopolitics. I hope it gives those who are afraid of the dangers of AI hope and a concrete direction to push for, and I hope it convinces those who are not convinced of the dangers that working in this direction is still beneficial and safer for the world than the alternative.
The essay starts from where AI stands. It is developing very fast, and it will likely become much stronger in the near future. How fast exactly, no one knows – the serious estimates run from a few years to a few decades. What is not in doubt is the direction. As AI gets more powerful, the number of groundbreakingly positive applications explodes. So do the risks. The essay sets out the evidence AI safety research has gathered for the catastrophic ones – the loss of control over AI, and with it over our future, and catastrophic misuse, such as AI-assisted creation of bioweapons – alongside harms already arriving: damage to mental health, especially in children, and rapid disruption of work. And it argues that there is a third catastrophic risk, one that comes less from the technology than from the enormous power it carries: a technology believed to decide future economic and military leadership raises the stakes between nations with every step it takes, and pressure like that, applied to rivalries that already exist, can escalate them into war.
It then looks at the response so far. Much has moved in the past months. The risks and the scenarios are discussed openly. More than a thousand researchers inside the leading labs have asked their government for the tools to pace AI development, the heads of those labs have called for a slower pace, and more than thirty heads of state and government have asked for an international institution that can set standards and verify them. But the frontier labs are still racing, and so are the largest powers. For the companies it is an economic race – for money, for power, and for being the first to build something that outthinks us. For the great powers it is a geopolitical one, in which falling behind in a technology that amplifies every other form of power looks like ending up at the mercy of whoever leads. Follow either race in the direction it is heading and it points toward outcomes no nation would choose: escalation between nuclear-armed rivals, systems advancing faster than anyone can keep them under control, and no time left for the safety work that would be needed. That is not for lack of good will. The essay traces it to dynamics – economic, geopolitical and psychological – that push every actor toward speed and away from caution, however much any one of them might prefer to slow down.
These dynamics can seem inevitable. The essay argues they are not: they are built by people, and they change when the counter to them is coordinated and strong enough. There are ways to control parts of them. Companies can be dissuaded from taking risks by strong liability and accountability rules that punish any safety failure severely. But other incentives can overrule both the caution of companies and the willingness of their governments to enforce such rules, or to pass them in the first place. So the counter has to take all of these factors together, and it has to do three things: find safe paths through timelines no one can predict, choose mitigations that are truly sufficient rather than merely comfortable, and counteract the dynamics that drive the current race. The rest of the essay works out the principles and the mechanisms that let us navigate increasingly powerful AI along such a path, with one fundamental shared goal and six principles at its base.
The fundamental goal the essay starts from is one nearly every person on earth already shares: we will not gamble with our children's lives. It means that we will search for the paths that reduce the risk of human extinction, or of an equally bad dystopia, to as near zero as we can make it. It means that we will not accept risking the lives of the ones we love for the promise of potential scientific or economic benefit.
The first principle, and the main one I want to highlight here, is clarity is power. Many arguments are made for why safeguards are unnecessary, too much of a competitive disadvantage, or why contracts between nations defining common rules are infeasible, and some of them sound convincing at first. To differentiate which of these arguments hold, and whether the paths they lead to are too dangerous to accept, requires carefully analysing the overall picture, despite the ever present uncertainty of AI progress. Moreover, this analysis needs to be clearly communicated and explained to the people and countries of the world, such that everyone can contribute to finding solutions and use their individual power to push for them.
The framework I propose has two parts.
Part 1 is an institution to coordinate a unified scientific process that maps out the different scenarios of how this technology and the geopolitics around it could unfold, that defines joint rules of what we should not do and what is necessary to keep all nations safe. This process has to be international, as all nations should be able to contribute to it as well as act on what it finds. It can be supported by national and continental institutions, legislation and standards, but its most important contributions would be global. Such an institution would need a mandate that is not limited to any specific field but takes into account the full picture of economics, warfare, competition and human rights; full access to all AI development efforts; experts from different fields and from all over the world; and a process that allows the world's scientific community to effectively contribute to it. This is where the clarity the first principle asks for would be produced. The decisions themselves remain with nations; what the institution gives them is the shared picture to decide against. Different processes akin to this one have started, the most notable of which are the UN scientific AI panel, China's WAICO effort and the EU AI Act. But while they are each valuable, they are very far away from what I describe above.
Part 2 is a set of contracts between nations that bind them to rules informed by the above process, that cannot be broken without considerable cost. Given the extreme economic and geopolitical advantages this technology promises, and the fear that an adversary could break the rules and reach them first, rules and standards on their own would be broken easily. The path therefore requires mechanisms that would slow down and disadvantage any country that breaks these rules, in such a way that breaking the contracts becomes very hard. And we need to ensure a balance of economic and geopolitical power that never puts nations into a position where it would be necessary for them to break the contracts.
There are different mechanisms for how to design these contracts to hold. One proposed mechanism is mutually assured compute destruction, a concept under which each nation would make its compute infrastructure intentionally and mutually vulnerable to its adversaries, such that a breakdown of the contracts would lead to the destruction of a large portion of both or all parties' compute resources, but would not necessarily result in war. Another, complementary strategy would be to keep the world's advanced compute supply chains intentionally bottlenecked, such that in a breach of contract the fast build-up of more compute resources would not be unilaterally possible. Another additional element could be predefining globally coordinated sanctions on anyone breaking the contract.
What all of these mechanisms require are: (1) some level of transparency and auditing between nations about AI research and development efforts; (2) compute tracking, to ensure all frontier efforts are fully monitored; (3) global rules on AI development and global safety standards, set out by the international institution proposed above, such that the rules of the contract are always clear but can evolve in a careful and well-defined manner as we gain more knowledge about AI.
At the moment these contracts might seem infeasible due to missing political will, but there are good reasons why they might become feasible in the near future. As AI develops, the power and the risks of this technology will become increasingly obvious to different nations and the general public. As we discuss the different options, the different paths and where they lead will be increasingly understood. Questions that can yet be dodged or answered loosely, without explaining the entire picture, will come under increasing pressure to be answered.
As of yet, there are to my knowledge no good answers for how a race without such contracts and internationally coordinated rules would avoid (a) geopolitical escalation due to ruthless and eventually existential competition between nations, or (b) catastrophic safety incidents, like loss of control over too intelligent systems, due to not having enough time for safety measures and their development under an all-out race. That is why the pressure described above will be hard to deflect.
Furthermore, the public in different nations will likely be increasingly in favour of controlled development, as the immediate benefits of AI are likely few, with larger benefits taking time to develop and be distributed, while the economic disruption due to AI development and ever increasing geopolitical tensions could be immediate, driving existing issues such as the cost of living crisis to extreme levels.
Lastly, despite the great promises of AI, the real value in our world lies in the stories of our lives, and protecting that is far more important than racing for some utopia that is just as likely to be a dystopia. So as it gets clearer where this might be going, the world will make its voice heard.
Now, despite my optimism, it is important that we create something like the framework above as soon as possible, in order to avoid both small and major AI-driven catastrophes, and to avoid steering ever deeper into dynamics of geopolitical competition and ever faster AI development that could overpower our ability to build such a framework in time.
Sections 5.7 and 5.8 of the essay lay out concretely how we could steer the world toward this safer path, and what each group of actors can do. At a high level, the first step is to make the creation of a global AI governance framework a major political priority and goal within national governments. The heads of state and government who called for an international institution have begun this, but it is not yet a priority that countries actually act on. There will be resistance to such efforts, but countries should bind together to create a joint position and to start the work on creating proposals for a detailed framework. People in different countries should demand clarity about their country's plan and position on different scenarios, and push for plans that manage the risk in a way that gives them confidence in their security.
Beyond global governance, there is work individual countries can do without waiting for anyone: hold AI companies accountable, with very strong incentives to pursue safety far more strongly and diligently than they currently do; where they host AI companies, keep full transparency into the everyday developments at these companies and be ready to intervene whenever something might endanger their citizens; and create, within their own governments, an AI safety institute that can at the least give them country-specific advice on AI, AI development and coordination with the international community.
The international research and expert community should focus on clearly laying out analyses of how different scenarios might play out. So far only a few scenarios are explored, and many of the "claimed positive" scenarios, laid out for example by AI CEOs, are not explored in a geopolitical context or at any level of detail. Furthermore, mechanisms for creating geopolitical balance and coordination of AI development should be explored heavily and discussed with policymakers in the loop. Technologies such as verification tools and model safeguards during training, evaluation and deployment should be developed as much as possible.
Beyond the above measures, which are targeted at solving the problem of the dynamics, there are many more individual measures that countries should consider, some of which are also laid out in section 5.7 of the full essay. It is important to point out that AI touches many risks, and almost all of these should get staffed experts to monitor them and suggest appropriate legislation.
Finally, I want to end on the one point I left open within the framework: the necessary balance of power between nations, to avoid getting any nation into a situation where it needs to consider breaking the contract. Historically, it has made sense for nations to compete to ensure their own security and to gain as much of the globally scarce resources for their population as possible. In the future, if we develop AI correctly, the wealth in the world will increase more than enough to share it around the world, so competing between nations for wealth will become increasingly unnecessary. What is more, aiming towards a shared future for all nations creates the security that, in a world of exponentially increasing technological power, would otherwise be impossible. It should be the international position and vision of every country to work towards such a shared future, while keeping the necessary national and economic security intact along the way.
Full essay, with the principles, the mechanisms and what each group of actors can do: https://beatthedynamics.org
I would be glad for your critique.
