Crossposted at zachgroff.com but edited for an audience already familiar with EA.

When I was a sophomore in college, I stumbled into a lecture by the economist I now work for, Dean Karlan, on the use of randomized controlled trials (RCTs) in economic development. I was inspired and excited. I’d always looked for ways to good with my life, and as a math nerd I was delighted to find a way to do good using numbers. I was also convinced: it seemed obvious to me that given the choice between doing something an RCT shows works and something that doesn’t, you should obviously do the former – and the difference could be so big that doing more RCTs was vital.
I’m still convinced, but I’m also aware of the drawbacks of focusing too narrowly RCTs and think they are quite large in the case of the animal rights movement. See, there are some things where an RCT doesn’t make sense. If you’re testing whether a malaria pill indeed prevents malaria in those who receive it, an RCT is a great idea. If instead you are trying to lobby government ministers, an RCT may not work.
You might be able to draw up a list of government ministers large enough to randomize among them depending on who you’re targeting, but if it’s just one or two, it won't work. Even if there are hundreds of relevant officials, if the lobbying effort is high-risk, high-reward – that is, if winning over one out of every 1000 ministers is enough to make it worthwhile – then an RCT probably won’t do the trick.
What I also left out is that even the RCT on the malaria pill will miss whatever we don’t measure. If, for instance, distributing a malaria pill leads government ministers to become complacent and stop supporting government anti-malaria programs that do far more good, we won’t pick that up.
So the natural limitation of RCT is that we can't use them to study interventions that are sufficiently high-risk, high-reward or where there is only a very small number of relevant individuals. Plus, we can miss effects that extend beyond the individuals studied and the measurements we took on them.
Given so, many people have criticized RCTs on the grounds that they result in tunnel vision that ignores institutional factors (I think this is the best such collection). I don’t think it’s fair to denounce RCTs as inaccurate or a bad way of studying the same things that had previously been studied in less-rigorous ways, but I think these arguments do get something right on the meta level: if RCTs are all we do or very close to all we do, then we will probably miss a lot of things.
I've become more sympathetic to institutional approaches to global poverty than I used to be. Many of my experiences in Ghana (and in Peru a few years back) made this more concrete for me. A banker I spoke to there went on a several-minute monologue about how many problems the lack of a national ID system creates. People can have four different IDs, and they can easily register for each one with a different name, making them extremely difficult for banks and the government to track. This has more costs than you would realize.
There are also softer, less visible institutions than government IDs that can have outsized effects, like education or even general expectations of drivers. I would probably have $50 more in my wallet if I did not point out mistakes on bills there that were in my favor. Meals are almost always under $10 in Ghana so that gives you a sense of how large and common these mistakes are.
Education, health, and many other institutions stack up in many ways to create what economists call transaction costs – what are essentially little taxes on everything you do that lead to less getting done.
Institutions may very well be an important driver of poverty, but the reason I'm somewhat skeptical of focusing on the more complicated among them is that they entail complex design issues for Westerners. The history of outsiders trying institutional change is spotty. That’s not to say it’s hopeless. Yet in general, it seems to me we’re not great at sophisticated design questions for developing countries that have much to do with local cultures and local politics, so we should focus on softer institutions or on supporting people in developing countries with the resources to fix institutions themselves. RCTs do give examples of how to affect some of these softer institutions, like education and healthcare. So I generally tend toward wanting to support those.
In the case of animal rights, I think the dynamic is very different. Though animal rights activists in the U.S. are a rarefied bunch – largely white, more educated than average, quite a bit more left-wing than average, and a lot more likely to be atheists – we are still more able to get a pulse on the culture we are working with and the design questions here.
(On a parallel note to the question of institutional design in other countries, the animal rights movement would probably be pragmatic to focus on elevating voices in specific marginalized communities to advocate there rather than sending outsiders in to advocate.)
There are a few other differences, I would say, with animal rights. One is that I’m somewhat pessimistic on the ability to change people’s minds enough on an individual level to be able to get a good reading of that. Public opinion change seems to come in spurts, making social change in general a high-risk, high-reward strategy. The literature on persuasion and advertising suggest that persuading individuals on their own is very, very hard to do. RCTs done by animal rights groups suggest that this is true as well in the case of the animal rights. I think it’s unlikely we can get sufficient power for an RCT to pick up much persuasion from anything in activists’ toolkits. (I would love to be proven wrong!)
On another note, I think there is more of a path forward for institutional change in the case of animal rights than in the case of global poverty. Of course, that’s not to say we lack any understanding of how to change the institutions that make a country rich or poor. Most of the issues, though, are again complex design questions about how to make a country’s leaders pay attention to the right things – questions that are especially vexing for outsiders.
My sense, though, is that there is a pretty powerful consensus that building a nonviolent protest movement can be highly effective. I've been reading histories of various social movements - not just civil rights and gay rights but things with varying track records like environmentalism, prohibition, and disability rights. It's hard to come away without thinking that movements rise and fall based on their extent of grassroots mobilization. There are issues with external validity and other questions about all of their work, but Erica Chenoweth, Sydney Tarrow, Doug McAdam and many other social scientists seem to concur that a densely connected and confrontational movement is highly effective for changing institutions. The counterargument seems to be a body of psychological literature showing that people get upset when they're confronted, but nobody ever denied this. The claim instead is that in the complex system of civil society a number of other mechanisms overwhelm that effect. Of the history of social movements I've read so far that's the conclusion I've come away with, though I'm working on reading more.
Of course, this is not an RCT – history is biased in many ways, as are disciplines like sociology that depend on it when looking at historical examples. We all look far more at successful movements than failed ones Moreover, even if we had literature on both varieties, we wouldn’t know for certain whether differences in tactics are actually responsible for differences in outcomes or the result of some outside force – say, that once a movement gains popular support it succeeds more and people start protesting more, without any relation between the two.
The careful historian can deal with this issue in ways that minimize it, though. Looking at the mechanics of a movement – not just when did protests happen and when did specific changes happen but how specific actors responded to each other in a more fine-grained way – can shed a bit more light on this. Quasi-experimental methods that simulate an experiment, while rare, can also help. If comparing movements while holding certain factors constant yields the same insight repeatedly, that’s also suggestive that there’s something going on. I can’t remember which sassy Facebook friend of mine said it, but while it’s true that correlation does not imply causation, it’s also true that correlation correlates with causation.
All of that is to say that yes, I think we have some sense of how to do institutional change for animal rights. I think we need more of a sense (and maybe some RCTs would work here, although I’ve had a hard time thinking of good proposals) of exactly what best mobilizes activists, pressures institutions, and triggers public controversies. But I think domestic change for the rights of an oppressed group involves a very different approach from work on global poverty.