This post presents the executive summary from Giving What We Can’s impact evaluation for 2025. At the end of this post we share links to more information, including the full report and...
I used AI to fix transcription errors, rerrarange the ideas, and suggest tweaks to the title and some sentences.
Three of the most exciting projects to come out of EA in recent years are, in a vague sense, CEA spinouts:
* Kairos is directly a spinout of CEA and now handles most support for university AI safety groups. Basically everyone I've found who knows them is really excited about what they do
* NEST is an opinionated ideas-fi...
Why aren't we engaging in direct action (including civil disobedience) to pause AI development?
Here's the problem:
Yudkowksy: "Many researchers steeped in these issues, including myself, expect that the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die."
Here's one solution:
FLI Open Letter: "all AI labs...immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium."
Here's what direct action in the pursuit of that solution could look like (most examples are from the UK climate movement):
Picketing AI offices (this already seems to be happening!)
Mass non-disruptive protest
Strikes/walk-outs (by AI developers/researchers/academics)
Slow marches
Roadblocks
Occupation of AI offices
Performative vandalism of AI offices
Performative vandalism of art
Sabotage of AI computing infrastructure (on the model of ecotage)
Theory of change:
In the words of Martin Luther King Jr., activists seek to "create such a crisis and foster such a tension that a community...is forced to confront the issue". Activists create disruption, gain publicity, generate (moral) outrage, and set an agenda; they force people – civil society, companies, governments – to think about an idea they weren't previously thinking about. This in turn can shift the Overton window, enact social change, and lead to political/legislative/policy change – e.g. a government-enforced moratorium on AI development.
Final thoughts:
AI-focused direct action on the model of climate activism currently seems extremely neglected and potentially highly effective. As a problem, the threat from AI is plausibly both more important and more tractable than climate change: a government-enforced global moratorium on AI development seems easier to achieve than a government-enforced global moratorium on e.g. issuing new fossil fuel licences, because there is arguably less to lose and the companies who are developing AI are smaller in number and (currently?) less powerful than e.g. fossil fuel companies.
Some potential reasons why not were posted here:
https://www.lesswrong.com/posts/o8nPDZmmiLQhi6fwt/support-me-in-a-week-long-picketing-campaign-near-openai-s
I think this is potentially a unilateralist-curse-type situation, so people should be careful before they engage in these types of disruption.
Thanks for the link – very helpful. I'm surprised by how unpopular the suggestion of an OpenAI picket is on LW.
To be clear, is your suggestion that engaging in AI-focused direct action could lead to a unilateralist's curse-type situation in which one government (presumably a goodish actor) pauses AI development, leaving others (presumably worse actors) to develop AI more easily?
If we could create a global AI-focused movement that would pressure governments simultaneously into coordinating a multilateral moratorium on development, would you support that?
No, I mean unilateralist curse in a different way:
Let's say lots of LessWrong people think about whether to go blockade the OpenAI offices while holding signs and yelling at people that AI will kill everyone. Most of the people conclude that this is a bad idea because there could be large negative consequences (potentially: OpenAI engineers becoming hostile to safety efforts, polarization, spreading an inaccurate picture of the problem, the wrong memes (terminator, killer robots) go mainstream, makes AI Alignment look crazy, etc.). However, one small minority of LessWrong people could think otherwise and that the effects are positive, and go on and do blockade the OpenAI offices.
What one should do in such a situation: coordinate with a larger group of people, think more carefully whether this is a good idea and listen to other people's argument, etc.
I see – and I presume you would agree with the majority of OpenAI people in this situation (i.e. direct action is a bad idea)?
Would you say the same thing about direct action taken against fossil fuel companies?
I think protesting/blocking fossil fuel companies is different and less of a unilateralist curse situation. For example, there is wide elite/expert agreement that more CO2 in the atmosphere is bad. We do not have that for the extinction of humanity due to AI. There also have been many protests against fossil fuel already, so additional protest is less likely to cause serious downsides or set the tone for future attempts to solve the problem. The nature of the problem is also different: incompetent political solutions to solve global warming often still help reduce CO2 somewhat, but the same might not be true for AI Notkilleveryoneism.
I am not sure whether "direct action" (imo a terrible name btw if the theory of change is indirect) against AI would be a good idea but lean against it currently.
We don't need to believe that AI will lead to human extinction to advocate for a moratorium on AI development. Karnofsky outlines a number of ways in which TAI could lead to global catastrophe here; and this 2021 survey of 44 AI risk researchers found the median estimate of existential risk was 32.5%. The risk from AI is a huge problem.
Do you think that climate protest is more harmful than helpful when it comes to solving the climate crisis?
This is a good point – but that's an argument for competent political solutions, not no political solutions (which is roughly what we have at the moment I think?).
Learn more here!