AI safety is also a behaviour change problem (and that means it needs social scientists)

TL;DR

• Much of AI safety depends on changing what governments, labs, engineers etc do. That makes it, in large part, a behaviour change problem.
• There are decades of evidence on how to change behaviour, including at the level of institutions and systems, and AI safety mostly isn’t drawing on it.
• Two common mistakes are relying on raising awareness (the knowledge deficit model) and assuming x-risk interventions can’t be evaluated.
• The field needs social science expertise in strategy, grantmaking and evaluation, as well as routes in for social scientists who want to help.

I’m a behavioural scientist. I was Deputy Director of Experimental Research at the Yale Program on Climate Change Communication, and I now lead impact measurement across 15 country offices as Head of Impact at an international NGO^[1]. I’ve had a few conversations recently with (fantastic, talented!) people in AI safety, that have made me realise how much the field is missing from social science. For example, several people have repeatedly suggested the need for new interventions focused on public education. Their reasoning is that if people knew more about AI safety they would take action, and so would governments. A couple of other people were reluctant to evaluate the x-risk interventions they were working on, because they assumed that they couldn’t be evaluated. These are understandable assumptions. Conservation held them for a long time too, and social scientists have been working on both problems for decades.^[2] 

Currently it feels like there are three tracks in the field: technical, governance, and “generalist”^[3] (which in practice means ops). I’d argue that much of AI safety depends on changing what governments, labs and publics do, and there is a whole field that studies how to do that. Irving & Askell gets cited as proof the field values social science, but it basically just described a handful of roles for computational cognitive scientists, as Vael Gates has pointed out. Emilio recently proposed a mentored research fellowship to bring more social science talent into AI safety, which I strongly hope gets funded. As he says, the world needs researchers to “study AI’s effects on people and institutions”. AI safety also needs social science expertise to reach its own goals.

Behaviour change for AI safety

So many societal problems are at their heart a behaviour change problem - this is a reckoning that biodiversity conservation^[4] went through recently, and I think it applies in part to AI safety also. I’ve argued before that conservation should learn from how public health and international development approached behaviour change, and AI safety can learn from all three. Don’t get me wrong - there are 100% massive technical problems we still need to solve in AI safety! But operationalising those solutions will mean targeting specific actors and behaviours. For example:

• governments: legislate, fund enforcement, coordinate internationally
• lab leadership: adopt safety frameworks and keep to them when it’s costly
• engineers: build safety culture, raise concerns
• compute providers and investors: due diligence, conditions on access
• publics: support regulation and vote on it; choose which products to use

Each one of the above behaviour change challenges has barriers and levers that can be diagnosed, designed for and tested.

An example: whistleblowing

Let’s take one example, the work of AIWI, who support whistleblowers in AI labs. We can apply the COM-B model (Michie, van Stralen and West 2011) to understand why people do or don’t whistleblow. This model posits that there are three essential components for behaviour change to occur: the capability (the physical and psychological ability to change), the opportunity (a physical and social environment that is conducive to change), and the motivation (positive personal beliefs and desires towards change). For whistleblowers, relevant barriers^[5] might span:

• Capability: people aren’t sure what counts as a legitimate concern, which channels exist, or what protection they’d have.
• Opportunity: there are very few trusted, AI-specific places to get advice, and the internal channel may be part of the problem.
• Motivation: career and equity risk, loyalty to colleagues and the mission, and the very human reasoning that someone more senior must already know.

Interventions that use behavioural models tend to be more successful. Combined with primary research, a model like this would help identify the best ways to support potential whistleblowers.

The knowledge deficit model

Unfortunately, it’s a super common mistake for people to rely on the knowledge deficit model when it comes to behaviour. This is basically the assumption that people just don’t know enough, and if we give them the correct information, they’ll act. A wealth of evidence shows that awareness and knowledge rarely translate into behaviour on their own. One great example is D.A.R.E., a programme where police officers taught schoolchildren about the dangers of drugs. It was the most widely used school drug prevention programme in the US, but meta-analyses found it had little to no effect on drug use. Knowing was not enough^[6]!

Sadly, I have many more examples, including a (really cool) intervention improving learning, but not behaviour. Education can have an impact, but combining multiple interventions (like education, prompts, and feedback) tends to be more effective at creating positive behaviour change. For AI safety, this means explainer-heavy strategies need theories of change that go beyond awareness. An AI risk explainer video should be judged on what viewers do afterwards, as well as what they learn.

Message testing is one place where this is starting to change. I’ve run message tests myself, including an experiment with almost 58,000 people in 23 countries, which found that climate messages increased support for climate action. In AI safety, Seismic Foundation and colleagues tested AI risk messages with over 1,000 people, and found that risks to jobs and children mobilised people more than existential risk did. Another survey experiment with 1,200 Americans found that fear raised perceived risk the most, but a hopeful message showing that regulation could work built more support for regulation. That same Seismic paper also notes that the public broadly backs strong AI regulation, but ranks it low against other issues. The concern is already there, and it isn’t turning into action.

This is a promising start, but most message tests measure attitudes, knowledge and intentions, which often don’t lead to behaviour. In one of my experiments, messages encouraging people to talk to friends about plant-based eating raised their intentions to do so, but didn’t move them to the next stage of preparing to act. Wherever possible, tests should include a behavioural outcome as well, like signing a petition, contacting a representative or sharing information^[7].

Institutions and social movements

The previous paragraphs focused more on individual behaviour, but social science also has a lot to say about institutional and systems change. For AI safety, that means looking beyond individuals to the institutions and incentives around them. Most AI governance work happens inside governments and think tanks. That work is valuable, but there is also a large research literature on how social movements create change from outside. AI safety already has groups running protests and public campaigns, and they can draw on a solid evidence base. Experiments show that extreme protest actions tend to reduce public support for a movement, because observers see them as immoral. But that backlash can be short-lived: after Animal Rising disrupted the Grand National, the negative effects on public attitudes had faded six months later, and the protest boosted media attention and mobilisation. A radical faction can also increase support for moderate organisations in the same movement, by making them look more reasonable in comparison (a positive radical flank effect). Media coverage matters too, as the way outlets covered the Grand National protest was linked to how much people supported the activists. Overall, the picture is more nuanced than either side of the “is disruption good?” debate tends to assume.

Evaluation and unknown unknowns

There’s a general problem of the “unknown unknowns” -  when you don’t have prior expertise, it’s easy to assume something can’t be done at all. This is particularly true for evaluation. I’ve written before about the need for stronger evaluations of AI safety talent programmes. I suggested a regression discontinuity design, but there are tons of other methods that could be used more broadly in the field (including for policy and advocacy work). These include contribution analysis, process tracing (including Bayesian process tracing, which lets you put numbers on how much each piece of evidence should update you), outcome harvesting and theory-based evaluation. I’ve used these kinds of methods myself, looking at why demand for ivory fell in Japan. The advocacy evaluation literature is a good place to start.

A lot of AI safety work focuses on x-risk, and it may seem like that makes it impossible to evaluate because well, either we go extinct or we don’t! However, every intervention should have a rigorous, specific theory of change, with the intervening steps and causal mechanisms clearly set out. We can test each of these steps to build up an evidence base for how well we’re reducing x-risk overall. Otherwise, funding goes to things nobody is learning from, and avoidable basic mistakes get repeated.

For example, take a campaign pushing frontier labs to adopt safety frameworks. The theory of change might be that public and investor pressure leads a lab to publish a framework, the framework contains specific, checkable thresholds, and the lab then sticks to them when a model crosses one. Each step leaves evidence: whether frameworks appear after campaign activity, whether their thresholds are concrete or vague, and whether deployments are delayed or modified when evaluations flag risks. Animal advocacy learned this the hard way with cage-free commitments: securing a pledge and getting it implemented are two separate outcomes that need tracking separately.

Building on the theory of change point above, participatory systems mapping is another underused tool. It brings stakeholders together to define the problem, agree the system’s boundaries, and map causal links to find leverage points. Applied to frontier lab governance, for example, it would show where interventions reinforce or undercut each other, which no single organisation’s theory of change can. It can also surface the gap between what stakeholders say they’re aiming for and their more pressing priorities (Stroh 2015). A lab may be publicly committed to safety, for instance, while its most pressing concern is staying at the frontier. I haven’t come across an AI safety organisation that has run one with outside experts. If yours has, I’d love to hear how it went, and if you’re thinking about trying one, get in touch.

What I’d like to see

A few concrete changes would help:

• Social scientists on grant panels and strategy reviews.
• Behavioural expertise involved from the design stage of comms and advocacy strategies.
• Evaluation built in from the start.
• Hiring for disciplinary expertise, beyond hiring social scientists into ops roles.

To sum up, the field’s hardest problems include human ones, and there are decades of evidence on how to work on them. Please start bringing social scientists into the conversation. If you’re designing an intervention and want a behavioural science sounding board, message me!

1. ^{^}

    Views my own! Disclosure: I’m also an author on several of the papers linked in this post, including the systematic review of conservation behaviour change interventions, the Grand National protest study, and the reviews on social movements and climate activism.

2. ^{^}

   Important disclaimer: social science is not a monolith, even if I seem to be referring to it that way. Other social scientists may very well disagree with some of my takes. That’s fine, the important thing is they should have theories and evidence to back them up! We might have different hypotheses, but we agree on how to test them. And note, my experience is that it’s much easier to involve a social scientist than for a technical person to attempt to learn social science methods - they involve more than just surveys, and far more goes into writing a good survey than you may assume!

3. ^{^}

    Or called fieldbuilding

^{^}

 I love this sentence: “Conservation of biodiversity is above all else an exercise in human persuasion”. (I am an author on that paper but I promise I didn’t write that sentence)

^{^}

 Theoretical barriers - I haven’t actually done primary research on this

^{^}

 Instead, it changed social norms around the prevalence of drug use in referent groups (some important concepts from behavioural science)

^{^}

 I actually think on the whole these aren’t fantastic proxies, but they’re a low bar!

Show all footnotes