For example, there is wide elite/expert agreement that more CO2 in the atmosphere is bad. We do not have that for the extinction of humanity due to AI.

We don't need to believe that AI will lead to human extinction to advocate for a moratorium on AI development. Karnofsky outlines a number of ways in which TAI could lead to global catastrophe here; and this 2021 survey of 44 AI risk researchers found the median estimate of existential risk was 32.5%. The risk from AI is a huge problem.

There also have been many protests against fossil fuel already, so additional protest is less likely to cause serious downsides or set the tone for future attempts to solve the problem.

Do you think that climate protest is more harmful than helpful when it comes to solving the climate crisis?

The nature of the problem is also different: incompetent political solutions to solve global warming often still help reduce CO2 somewhat, but the same might not be true for AI Notkilleveryoneism.

This is a good point – but that's an argument for competent political solutions, not no political solutions (which is roughly what we have at the moment I think?).

I am not sure whether "direct action" (imo a terrible name btw if the theory of change is indirect) against AI would be a good idea but lean against it currently.

Learn more here!

I see – and I presume you would agree with the majority of OpenAI people in this situation (i.e. direct action is a bad idea)?

Would you say the same thing about direct action taken against fossil fuel companies?

Thanks for the link – very helpful. I'm surprised by how unpopular the suggestion of an OpenAI picket is on LW.

To be clear, is your suggestion that engaging in AI-focused direct action could lead to a unilateralist's curse-type situation in which one government (presumably a goodish actor) pauses AI development, leaving others (presumably worse actors) to develop AI more easily?

If we could create a global AI-focused movement that would pressure governments simultaneously into coordinating a multilateral moratorium on development, would you support that?

Why aren't we engaging in direct action (including civil disobedience) to pause AI development?

Here's the problem:

Yudkowksy: "Many researchers steeped in these issues, including myself, expect that the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die."

Here's one solution:

FLI Open Letter: "all AI labs...immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium."

Here's what direct action in the pursuit of that solution could look like (most examples are from the UK climate movement):

Picketing AI offices (this already seems to be happening!)

Mass non-disruptive protest

Strikes/walk-outs (by AI developers/researchers/academics)

Slow marches

Roadblocks

Occupation of AI offices

Performative vandalism of AI offices

Performative vandalism of art

Sabotage of AI computing infrastructure (on the model of ecotage)

Theory of change:

In the words of Martin Luther King Jr., activists seek to "create such a crisis and foster such a tension that a community...is forced to confront the issue". Activists create disruption, gain publicity, generate (moral) outrage, and set an agenda; they force people – civil society, companies, governments – to think about an idea they weren't previously thinking about. This in turn can shift the Overton window, enact social change, and lead to political/legislative/policy change – e.g. a government-enforced moratorium on AI development.

Final thoughts:

AI-focused direct action on the model of climate activism currently seems extremely neglected and potentially highly effective. As a problem, the threat from AI is plausibly both more important and more tractable than climate change: a government-enforced global moratorium on AI development seems easier to achieve than a government-enforced global moratorium on e.g. issuing new fossil fuel licences, because there is arguably less to lose and the companies who are developing AI are smaller in number and (currently?) less powerful than e.g. fossil fuel companies.