Effective Altruism Forum
EA Forum

Hide table of contents
by Elliot Mckernon
5 min read 0

19

AI safetyPolicyUK policyAI governance
Frontpage

tl;dr: The new UK government will likely continue to balance encouraging AI innovation for public good against increasing regulation for public safety, with so-far rhetorical calls for stricter regulation than the previous government’s. Several reports have been published by the government and the UK AI Safety Institute, including the latter’s first technical report on model evaluation.

Previously on The UK’s AI Policy

Erstwhile Prime Minister Rishi Sunak took office in October 2022 and quickly announced a suite of new AI policies and plans. Broadly, Sunak's government saw AI as a stonking big opportunity for the UK's economy and society, via becoming a hub of AI development, revolutionizing public services, and providing $1 trillion in value for the UK by 2035. They described their regulatory approach as pro-innovation, calling for government oversight and, eventually, greater requirements on developers of frontier AI. 

Lol

However, Sunak did take AI safety and even existential risk seriously, saying: 

Get this wrong, and AI could make it easier to build chemical or biological weapons. Terrorist groups could use AI to spread fear and destruction [...] in the most unlikely but extreme cases, there is even the risk that humanity could lose control of AI completely [through] ‘super intelligence’. [...] I don’t want to be alarmist. And there is a real debate about this [...] But however uncertain and unlikely these risks are, if they did manifest themselves, the consequences would be incredibly serious.

To address these risks, the government organized the first international AI Safety Summit. You can read my summaries of the plans for and outcomes from the summit if you want more detail, but briefly, the summit resulted in the international Bletchley Declaration and a promise of ~£700 million over 7 years to the UK AI Safety Institute (née Frontier AI Safety taskforce, and not to be confused with the US or Canadian AI Safety Institutes, with both of whom they are partnered, nor the Japanese or Singaporean AI Safety Institutes, with whom they are not). The UK AISI called for input from labs and research institutes and started work on research topics like AI evaluations that we'll discuss below, with notable advisors and staff such as Yoshua Bengio, Ian Hogarth, Paul Christiano, and Matt Clifford. 

What do the new guys say?

Since then, the dramatic-in-a-British-way 2024 election reduced the tories’ seat count in parliament by two thirds and doubled Labour’s. Rishi Sunak has been replaced by Sir Keir Starmer. 

The new Labour government has indicated that they intend to regulate AI more tightly than the previous Tory government, while still encouraging growth of the AI sector and making use of AI in delivering their national missions. 

Before the election, Starmer stated the UK should introduce stronger regulation of AI, and Labour’s manifesto promised to introduce "binding regulation on the handful of companies developing the most powerful AI models". Other than planning to ban sexual deepfakes and outlawing nudification, we have little information on what this binding regulation would look like. 

Indeed, some recent statements seem to ape the previous government’s pro-innovation approach:

  • Labour’s manifesto commits to creating a pro-innovation regulatory body to update regulation, speed up approval timelines, and “co-ordinate issues that span existing boundaries” (???);
  • Investment into AI for the National Health Service;
  • A commitment to supporting AI development by removing barriers to new data centres and the creation of a National Data Library;
  • Revamping the department of science and technology to encourage AI development in the public sector.

What has the UK AISI and governmental friends been up to? 

AI Opportunities Unit

On the 26th July, the Secretary of State for Science, Innovation and Technology Peter Kyle stated that AI has enormous potential and that the UK must use AI to support their five national missions, while still developing next steps for regulating frontier AI. To do so:

  • An AI opportunities unit will be established within the Department for Science, Innovation and Technology;
  • Tech entrepreneur Matt Clifford will develop an AI opportunities action plan, to be submitted in September 2024;
  • The government will address "key AI enablers such as the UK’s compute and broader infrastructure requirements”.

Note that Kyle previously advocated for compelling AI developers by law to share test results with the UK AISI (rather than relying on existing voluntary sharing), though this hasn’t appeared in rhetoric or policy since.

King v Baron

In July during the King’s Speech, the government committed to legislating powerful AI by placing the UK AISI “on a statutory footing”, providing it with a permanent remit to improve safety while focusing specifically on developers of the most advanced frontier AI, rather than users or, as the EU AI Act does, AI developers more broadly. 

Despite widespread rumours of a fully formed and ready to go AI Bill, King Charles III didn’t mention any such bill. House of Lords member and nine-times-gold medal-winning paralympian, the Right Honourable Baron Holmes of Richmond, plans to re-introduce his proposed AI bill to the house. 

Reports from the UK AISI and AI Seoul Summit

The AISI released three reports in May this year:

  • Their first technical report on model evaluations. Their findings weren’t especially novel, but do demonstrate the government is developing at least some in-house testing chops: 
    • Several LLMs demonstrated expert-level knowledge in chemistry and biology;
    • Models solved high-school level cybersecurity challenges but struggled with university-level.
    • All tested models were highly vulnerable to basic jailbreak attacks, complying with harmful requests.
    • Models safeguards could be bypassed to elicit harmful information.
       
  • Their fourth progress report. 
    • They’ve open-sourced Inspect, a software library for assessing AI model capabilities.
    • They’ve set up a San Francisco office to collaborate with the US.
    • They’ve partnered with the Canadian AI Safety Institute.
    • Jade Leung has been appointed as Chief Technology Officer.
    • They’re continuing to focus on evaluating risk from frontier AI. 
    • They have a new program to increase societal resilience to AI risks.
       
  • The mammoth Interim International Scientific Report on the Safety of Advanced AI, commissioned by the 2023 summit and chaired by Yoshua Bengio, was published for the May 2024 AI Seoul Summit. The report’s too long to properly summarize, but along with some standard chatter about rapid AI progress and future uncertainty, and explanations of how AI is developed etc, the report notes:
    • AI is approaching human-level performance and will likely transform many jobs.
    • Evaluating GPAI remains a challenge; benchmarking and red-teaming are insufficient to assure safety. 
    • Technical safety approaches like adversarial training are helpful, but no current methods can guarantee safety for advanced general-purpose AI.
    • Experts disagree about likelihood and timelines of extreme risks like loss of control.
    • The trajectory of AI development will be shaped by societal choices.
    • Energy demands will strain electrical infrastructure.
    • Systemic risks include disruption of the Labour market, exacerbation of income inequality, concentration of power to few countries & companies (which increases risks from single points of failure), environmental harm, threats to privacy, copyright infringement
       
  • At the AI Seoul Summit itself, co-hosted by the UK and South Korea, 10 countries agreed to develop an international network of AI Safety Institutes and £8.5 million in funding grants for research on systemic AI safety was announced (delivered through the UK AISI  & partnered institutes). See here for other takeaways from the summit.

If you’re interested in a more in-depth analysis of existing AI regulations in the EU, China, and the US, check out our 2024 State of the AI Regulatory Landscape report. 

19

0
0
2

Reactions

0
0
2

More posts like this

Comments


No comments on this post yet.
Be the first to respond.
More from Elliot Mckernon
32
Reviewing the Structure of Current AI Regulations
3 authors
· · 16m read
1
1
52
Now THIS is forecasting: understanding Epoch's Direct Approach
3 authors
· · 23m read
2
2
52
Now THIS is forecasting: understanding Epoch's Direct Approach
3 authors
· · 23m read
2
2
View more
Curated and popular this week
LintzA
 ·  · 15m read
 · 
Cross-posted to Lesswrong Introduction Several developments over the past few months should cause you to re-evaluate what you are doing. These include: 1. Updates toward short timelines 2. The Trump presidency 3. The o1 (inference-time compute scaling) paradigm 4. Deepseek 5. Stargate/AI datacenter spending 6. Increased internal deployment 7. Absence of AI x-risk/safety considerations in mainstream AI discourse Taken together, these are enough to render many existing AI governance strategies obsolete (and probably some technical safety strategies too). There's a good chance we're entering crunch time and that should absolutely affect your theory of change and what you plan to work on. In this piece I try to give a quick summary of these developments and think through the broader implications these have for AI safety. At the end of the piece I give some quick initial thoughts on how these developments affect what safety-concerned folks should be prioritizing. These are early days and I expect many of my takes will shift, look forward to discussing in the comments!  Implications of recent developments Updates toward short timelines There’s general agreement that timelines are likely to be far shorter than most expected. Both Sam Altman and Dario Amodei have recently said they expect AGI within the next 3 years. Anecdotally, nearly everyone I know or have heard of who was expecting longer timelines has updated significantly toward short timelines (<5 years). E.g. Ajeya’s median estimate is that 99% of fully-remote jobs will be automatable in roughly 6-8 years, 5+ years earlier than her 2023 estimate. On a quick look, prediction markets seem to have shifted to short timelines (e.g. Metaculus[1] & Manifold appear to have roughly 2030 median timelines to AGI, though haven’t moved dramatically in recent months). We’ve consistently seen performance on benchmarks far exceed what most predicted. Most recently, Epoch was surprised to see OpenAI’s o3 model achi
Dr Kassim
 ·  · 4m read
 · 
Hey everyone, I’ve been going through the EA Introductory Program, and I have to admit some of these ideas make sense, but others leave me with more questions than answers. I’m trying to wrap my head around certain core EA principles, and the more I think about them, the more I wonder: Am I misunderstanding, or are there blind spots in EA’s approach? I’d really love to hear what others think. Maybe you can help me clarify some of my doubts. Or maybe you share the same reservations? Let’s talk. Cause Prioritization. Does It Ignore Political and Social Reality? EA focuses on doing the most good per dollar, which makes sense in theory. But does it hold up when you apply it to real world contexts especially in countries like Uganda? Take malaria prevention. It’s a top EA cause because it’s highly cost effective $5,000 can save a life through bed nets (GiveWell, 2023). But what happens when government corruption or instability disrupts these programs? The Global Fund scandal in Uganda saw $1.6 million in malaria aid mismanaged (Global Fund Audit Report, 2016). If money isn’t reaching the people it’s meant to help, is it really the best use of resources? And what about leadership changes? Policies shift unpredictably here. A national animal welfare initiative I supported lost momentum when political priorities changed. How does EA factor in these uncertainties when prioritizing causes? It feels like EA assumes a stable world where money always achieves the intended impact. But what if that’s not the world we live in? Long termism. A Luxury When the Present Is in Crisis? I get why long termists argue that future people matter. But should we really prioritize them over people suffering today? Long termism tells us that existential risks like AI could wipe out trillions of future lives. But in Uganda, we’re losing lives now—1,500+ die from rabies annually (WHO, 2021), and 41% of children suffer from stunting due to malnutrition (UNICEF, 2022). These are preventable d
Rory Fenton
 ·  · 6m read
 · 
Cross-posted from my blog. Contrary to my carefully crafted brand as a weak nerd, I go to a local CrossFit gym a few times a week. Every year, the gym raises funds for a scholarship for teens from lower-income families to attend their summer camp program. I don’t know how many Crossfit-interested low-income teens there are in my small town, but I’ll guess there are perhaps 2 of them who would benefit from the scholarship. After all, CrossFit is pretty niche, and the town is small. Helping youngsters get swole in the Pacific Northwest is not exactly as cost-effective as preventing malaria in Malawi. But I notice I feel drawn to supporting the scholarship anyway. Every time it pops in my head I think, “My money could fully solve this problem”. The camp only costs a few hundred dollars per kid and if there are just 2 kids who need support, I could give $500 and there would no longer be teenagers in my town who want to go to a CrossFit summer camp but can’t. Thanks to me, the hero, this problem would be entirely solved. 100%. That is not how most nonprofit work feels to me. You are only ever making small dents in important problems I want to work on big problems. Global poverty. Malaria. Everyone not suddenly dying. But if I’m honest, what I really want is to solve those problems. Me, personally, solve them. This is a continued source of frustration and sadness because I absolutely cannot solve those problems. Consider what else my $500 CrossFit scholarship might do: * I want to save lives, and USAID suddenly stops giving $7 billion a year to PEPFAR. So I give $500 to the Rapid Response Fund. My donation solves 0.000001% of the problem and I feel like I have failed. * I want to solve climate change, and getting to net zero will require stopping or removing emissions of 1,500 billion tons of carbon dioxide. I give $500 to a policy nonprofit that reduces emissions, in expectation, by 50 tons. My donation solves 0.000000003% of the problem and I feel like I have f
Recent opportunities in AI safety
20
Hiring pre-docs
Eva
· · 1m read
0
0
57
Quick nudge to apply to the LTFF grant round (closing on Saturday)
calebp
· · 1m read
7
7
47
Public Comment Invited on Artificial Intelligence Action Plan
PeterSlattery
· · 1m read
0
0
View more