James Özden and Sam Glover at Social Change Lab wrote a literature review on protest outcomes[1] as part of a broader investigation[2] on protest effectiveness. The report covers multiple lines of evidence and addresses many relevant questions, but does not say much about the methodological quality of the research. So that's what I'm going to do today.
I reviewed the evidence on protest outcomes, focusing only on the highest-quality research, to answer two questions:
1. Do protests work?
2. Are Social Change Lab's conclusions consistent with the highest-quality evidence?
Here's what I found:
Do protests work? Highly likely (credence: 90%) in certain contexts, although it's unclear how well the results generalize. [More]
Are Social Change Lab's conclusions consistent with the highest-quality evidence? Yes—the report's core claims are well-supported, although it overstates the strength of some of the evidence. [More]
Cross-posted from my website.
Introduction
This article serves two purposes: First, it analyzes the evidence on protest outcomes. Second, it critically reviews the Social Change Lab literature review.
Social Change Lab is not the only group that has reviewed protest effectiveness. I was able to find four literature reviews:
1. Animal Charity Evaluators (2018), Protest Intervention Report.
2. Orazani et al. (2021), Social movement strategy (nonviolent vs. violent) and the garnering of third-party support: A meta-analysis.
3. Social Change Lab – Ozden & Glover (2022), Literature Review: Protest Outcomes.
4. Shuman et al. (2024), When Are Social Protests Effective?
The Animal Charity Evaluators review did not include many studies, and did not cite any natural experiments (only one had been published as of 2018).
Orazani et al. (2021)[3] is a nice meta-analysis—it finds that when you show people news articles about nonviolent protests, they are more likely to express support for the protesters' cause. But what people say in a lab setting mig
A distinction I've found useful is "object-level" vs "social reality". They are both adjectives that describe types of conversation/ ideas.
Object-level discussions are about ideas and actions (e.g. AI timelines, the mechanics of launching a successful startup). Object-level ideas are technical, empirical, and often testable. Object-level refers to what ideas are important or make sense. It is focused on truth-seeking and presenting arguments clearly.
Social reality discussions are about people and organisations (e.g. Will MacAskill, Open Philanthropy). Social reality is more meta, more abstract, and less testable than object-level. Social reality refers to which people are influential/powerful (and what they think), how to network with people, how to persuade people.
I have found it very helpful to start labelling whether I'm in object-level conversation mode vs social reality conversation mode. It helps me notice when I'm deferring without having thought about it (e.g. "well, Will MacAskill says [x]" instead of asking myself what I think about [x]), or when I fall into a mode of chit-chatting about the who's-who of EA, instead of trying to truth-seek (of course, chit chatting sometimes is fine -- I just want to be intentional about when I'm doing it).
And social reality isn't necessarily bad, but it's helpful to flag when a conversation enters "social reality mode."
I do think it's good for many/more/most conversations to centre around the object-level. I am personally trying to move my ratio more towards object-level.
(This was a core theme of an Atlas camp I attended, which I found extremely valuable. The above definitions are loosely based on a message from Jonas, but I didn't run them by him before posting.)
This relates to a caveat in my recent post:
Part of me wants to flesh this thought out properly soon. But even this conversation is meta! And I'm trying to encourage/ focus more on object-level ideas. So do I write it? I'm not sure.