James Özden and Sam Glover at Social Change Lab wrote a literature review on protest outcomes[1] as part of a broader investigation[2] on protest effectiveness. The report covers multiple lines of evidence and addresses many relevant questions, but does not say much about the methodological quality of the research. So that's what I'm going to do today.
I reviewed the evidence on protest outcomes, focusing only on the highest-quality research, to answer two questions:
1. Do protests work?
2. Are Social Change Lab's conclusions consistent with the highest-quality evidence?
Here's what I found:
Do protests work? Highly likely (credence: 90%) in certain contexts, although it's unclear how well the results generalize. [More]
Are Social Change Lab's conclusions consistent with the highest-quality evidence? Yes—the report's core claims are well-supported, although it overstates the strength of some of the evidence. [More]
Cross-posted from my website.
Introduction
This article serves two purposes: First, it analyzes the evidence on protest outcomes. Second, it critically reviews the Social Change Lab literature review.
Social Change Lab is not the only group that has reviewed protest effectiveness. I was able to find four literature reviews:
1. Animal Charity Evaluators (2018), Protest Intervention Report.
2. Orazani et al. (2021), Social movement strategy (nonviolent vs. violent) and the garnering of third-party support: A meta-analysis.
3. Social Change Lab – Ozden & Glover (2022), Literature Review: Protest Outcomes.
4. Shuman et al. (2024), When Are Social Protests Effective?
The Animal Charity Evaluators review did not include many studies, and did not cite any natural experiments (only one had been published as of 2018).
Orazani et al. (2021)[3] is a nice meta-analysis—it finds that when you show people news articles about nonviolent protests, they are more likely to express support for the protesters' cause. But what people say in a lab setting mig
One thing I will say here that I think shouldn't be controversial:
At the very least the Cade Metz NYT on Scott fairly clearly did not give readers a misleading impression (whether or not it gave the reader that impression in a fair way.): the article does not state "Scott Alexander is a hardcore white nationalist", or even, in my view, really give people that impression. What it does give the reader as an impression is that he is highly skeptical of feminism and social justice, his community of followers includes white nationalists, and he is sympathetic to views on race on which Black people are genetically intellectually inferior. All these things are true, as anyone who reads Thorstad's blogpost can verify. But more importantly, while I understand not everyone reads Scott and his blog commentators religiously, all these things are fairly obviously true if you've followed Scott's writing closely. (As I have; I used to like it a great deal, before disagreement on exactly this stuff very gradually soured me on it.*) I think it is a failure of community epistemics that a lot of people jumped to "this is a smear" before really checking, or suspending judgment.
*I actually find this whole topic very emotionally upsetting and confusing, because I think I actually have a very similar personality to Scott and other rationalists, and seeing them endorse what to me is fairly obvious evil-I'm talking here about reactionary political projects here, not any particular empirical beliefs-makes me worried that I am bad too. Read everything I say on this thread with this bias in mind.
I identify with your asterisk quite a bit. I used to be much more strongly involved in rationalist circles in 2018-2020, including the infamous Culture War Thread. I distanced myself from it around ~2020, at the time of the NYT controversy, mostly just remaining on Rationalist Tumblr. (I kinda got out at the right time because after I left everyone moved to Substack, which positioned itself against the NYT by personally inviting Scott, and was seemingly designed to encourage every reactionary tendency of the community.)
One of the most salient memories of t... (read more)