The AI safety Summit, initiated by Rishi Sunak, will take place on Wednesday and Thursday. It attempts to convene international governments (including China), leading AI companies, civil society groups, and experts in research across two days.

I spent a bit of time trying to understand the intended outcomes and the summit's agenda and thought I wanted to share that here.

Objectives

The summit has five objectives:

• a shared understanding of the risks posed by frontier AI and the need for action
• a forward process for international collaboration on frontier AI safety, including how best to support national and international frameworks
• appropriate measures which individual organisations should take to increase frontier AI safety
• areas for potential collaboration on AI safety research, including evaluating model capabilities and the development of new standards to support governance
• showcase how ensuring the safe development of AI will enable AI to be used for good globally

Agenda

The Summit has the following agenda (copy-pasted).

Day 1

Understanding Frontier AI Risks (roundtable discussions)

Delegates will break out to discuss the following topics with multi-disciplinary attendees. Conclusions from each session will be published at the end of the summit.

1. Risks to Global Safety from Frontier AI Misuse
Discussion of the safety risks posed by recent and next generation frontier AI models, including risks to biosecurity and cybersecurity.

2. Risks from Unpredictable Advances in Frontier AI Capability
Discussion of risks from unpredictable ‘leaps’ in frontier AI capability as models are rapidly scaled, emerging forecasting methods, and implications for future AI development, including open-source.

3. Risks from Loss of Control over Frontier AI
Discussion of whether and how very advanced AI could in the future lead to loss of human control and oversight, risks this would pose, and tools to monitor and prevent these scenarios.

4. Risks from the Integration of Frontier AI into Society
Risks from the integration of frontier AI into society include election disruption, bias, impacts on crime and online safety, and exacerbating global inequalities. The discussion will include measures countries are already taking to address these risks.

Improving Frontier AI Safety (roundtable discussions)

Delegates will break out to discuss the following topics with multi-disciplinary attendees. Conclusions from each session will be published at the end of the summit.

1. What should Frontier AI developers do to scale responsibly?
Multidisciplinary discussion of Responsible Capability Scaling at frontier AI developers including defining risk thresholds, effective model risk assessments, pre-commitments to specific risk mitigations, robust governance and accountability mechanisms, and model development choices.

2. What should National Policymakers do in relation to the risks and opportunities of AI?
Multidisciplinary discussion of different policies to manage frontier AI risks in all countries including monitoring, accountability mechanisms, licensing, and approaches to open-source AI models, as well as lessons learned from measures already being taken.

3. What should the International Community do in relation to the risks and opportunities of AI?
Multidisciplinary discussion of where international collaboration is most needed to both manage risks and realise opportunities from frontier AI, including areas for international research collaborations.

4. What should the Scientific Community do in relation to the risks and opportunities of AI?
Multidisciplinary discussion of the current state of technical solutions for frontier AI safety, the most urgent areas of research, and where promising solutions are emerging.

AI for good – AI for the next generation (panel discussion)

Discussion on the immense opportunities of AI to transform education for future generations, followed by closing remarks by the UK’s Secretary of State.

Day 2

The Prime Minister will convene a small group of governments, companies and experts to further the discussion on what steps can be taken to address the risks in emerging AI technology and ensure it is used as a force for good.

In parallel, UK Technology Secretary Michelle Donelan will reconvene international counterparts to agree on the next steps.

The website says that "Conclusions from each session will be published at the end of the summit." so it's probably worth staying updated over the coming week. It feels a bit surreal, but this combined with initiatives such as Biden’s Executive Order on Safe AI, UN’s high-level advisory body on AI, and the UK Taskforce makes me much more hopeful that we might actually avoid an existential catastrophe from AI.