Hide table of contents

Across the community it is common to hear distinctions drawn between ‘broad’ and ‘narrow’ interventions - though less so lately than in 2014/2013. For some imperfect context see this blog post by Holden on 'flow-through effects'. Typical classifications people might make would be:

 

Broader


  • GiveDirectly or the Against Malaria Foundation, with the goal of general human empowerment

  • Studying math

  • Promoting effective altruism

  • Improving humanity’s forecasting ability through prediction markets


Narrower

  • Specialising in answering one technical question about e.g. artificial intelligence or nanotechnology, etc.

  • Studying 'crucial considerations'

  • Trying to stop Chris Christie becoming President in 2020

  • Trying to stop a war between India and Pakistan


All I want to do here is draw attention to the fact that there are multiple distinctions that should be drawn out separately so that we can have more productive conversations about the relative merits and weaknesses of different approaches. One way to do this is to make causal diagrams. I’ve made two below for illustrative purposes.

Below is a list of some possible things we might mean by narrow and broad, or related terms.

  • A long path to impact vs a short path to impact. Illustrated here:


  • Many possible paths to impact vs few possible paths to impact. Illustrated here.

  • The path functions in a wide variety of possible future scenarios vs path can function in only one possible (and conjunctive, or unlikely) future scenario. Illustrated amusingly here (though mixed in with other forms of narrowness as well).


  • The path has many necessary steps that are notably weak (that is, there’s a high probability they might not happen) vs the path has few such weak steps.


  • The path has a weak step that might not happen early on in its causal chain, vs path has a weak step that might not happen later in its causal chain.


  • The path initially has only few routes, but after one step opens up many paths, vs the path initially has many paths but after later on narrows down to few paths.

 

Other quick observations about this:

  • A ‘narrow’ approach on any of these definitions clearly has downsides, but could be compensated for with greater ‘force’ e.g. the magnitude of the impact along a single chain is very huge if the scheme works out.

  • It’s hard to know what should ‘count’ as a step and what doesn’t. How do we cleave these causal diagrams at their true joints? One rule of thumb for drawing out such diagrams in practice could be that a ‘step’ in the chain is something that is less than 99% likely to happen - anything higher than that can be ignored, though this too is vulnerable to arbitrariness in what you lump together as a single step.

  • I believe in the overwhelming importance of shaping the long term future. In my view most causal chains that could actually matter are likely to be very long by normal standards. But they might at least have many paths to impact, or be robust (i.e. have few weak steps).

  • People who say they are working on broad, robust or short chains usually ignore the major uncertainties about whether the farther out regions of the chain they are a part of are positive, neutral or negative in value. I think this is dangerous and makes these plans less reliable than they superficially appear to be.

  • If any single step in a chain produces an output of zero, or negative expected value (e.g. your plan has many paths to increasing our forecasting ability, but it turns out that doing so is harmful), then the whole rest of that chain isn’t desirable.

  • Most plans in practice seem to go through a combination of both narrow and broad sections (e.g. lobbying for change to education policy has a bottleneck around the change in the law; after that there may be many paths to impact).

  • I'm most excited about look for ways to reduce future risks that can work in a wide range of scenarios - so called 'capacity building'.


As they say, more research is needed.

 

Comments3


Sorted by Click to highlight new comments since:

Thanks Rob, I think this is a valuable space to explore. I like what you've written. I'm going to give an assortment of thoughts in this space.

I have tended to refer to the long- vs short- path to impact as "indirect vs direct", and the many-paths-to-impact vs few-paths-to-impact as "broad vs narrow/targeted". I'm not sure how consistently these terms are understood. Another distinction which comes up is the degree of speculativeness of the intervention.

There are some correlations between these different distinctions:

  • Indirect interventions have more opportunities to be broad than direct ones
  • Indirect interventions are typically more speculative than direct ones
  • but broad interventions are often more robust (less speculative) than narrow ones

I think it's typically easier to get a good understanding of effectiveness for more direct and more narrow interventions. I therefore think they should normally be held to a higher standard of proof -- the cost of finding that proof shouldn't be prohibitive in a way it might be for the broader interventions.

I'm particularly suspicious of indirect, narrow interventions. Here there is a single chain for the intended effect, and lots of steps in the chain. This means that if we've made a mistake about our reasoning at any stage, the impact of the entire thing could collapse.

Great clarification, thanks!

There are some correlations between these different distinctions:

  • Indirect interventions have more opportunities to be broad than direct ones
  • Indirect interventions are typically more speculative than direct ones
  • but broad interventions are often more robust (less speculative) than narrow ones

To sum up, the shallower and broader the causal map, the more rubust it is.

The outside view should in theory have input into how fragile we can build our inference chains. In practice it is likely very hard to establish a base rate due to the issue you bring up of not knowing where to carve. "Steps of uniform likelihood" isn't exactly an operation we can apply to a data set on past results. If we're stuck with expert judgement, that limits how many cases we can evaluate and how confident we can be in the results. Still better than nothing though.

Curated and popular this week
 ·  · 5m read
 · 
[Cross-posted from my Substack here] If you spend time with people trying to change the world, you’ll come to an interesting conundrum: Various advocacy groups reference previous successful social movements as to why their chosen strategy is the most important one. Yet, these groups often follow wildly different strategies from each other to achieve social change. So, which one of them is right? The answer is all of them and none of them. This is because many people use research and historical movements to justify their pre-existing beliefs about how social change happens. Simply, you can find a case study to fit most plausible theories of how social change happens. For example, the groups might say: * Repeated nonviolent disruption is the key to social change, citing the Freedom Riders from the civil rights Movement or Act Up! from the gay rights movement. * Technological progress is what drives improvements in the human condition if you consider the development of the contraceptive pill funded by Katharine McCormick. * Organising and base-building is how change happens, as inspired by Ella Baker, the NAACP or Cesar Chavez from the United Workers Movement. * Insider advocacy is the real secret of social movements – look no further than how influential the Leadership Conference on Civil Rights was in passing the Civil Rights Acts of 1960 & 1964. * Democratic participation is the backbone of social change – just look at how Ireland lifted a ban on abortion via a Citizen’s Assembly. * And so on… To paint this picture, we can see this in action below: Source: Just Stop Oil which focuses on…civil resistance and disruption Source: The Civic Power Fund which focuses on… local organising What do we take away from all this? In my mind, a few key things: 1. Many different approaches have worked in changing the world so we should be humble and not assume we are doing The Most Important Thing 2. The case studies we focus on are likely confirmation bias, where
 ·  · 2m read
 · 
I speak to many entrepreneurial people trying to do a large amount of good by starting a nonprofit organisation. I think this is often an error for four main reasons. 1. Scalability 2. Capital counterfactuals 3. Standards 4. Learning potential 5. Earning to give potential These arguments are most applicable to starting high-growth organisations, such as startups.[1] Scalability There is a lot of capital available for startups, and established mechanisms exist to continue raising funds if the ROI appears high. It seems extremely difficult to operate a nonprofit with a budget of more than $30M per year (e.g., with approximately 150 people), but this is not particularly unusual for for-profit organisations. Capital Counterfactuals I generally believe that value-aligned funders are spending their money reasonably well, while for-profit investors are spending theirs extremely poorly (on altruistic grounds). If you can redirect that funding towards high-altruism value work, you could potentially create a much larger delta between your use of funding and the counterfactual of someone else receiving those funds. You also won’t be reliant on constantly convincing donors to give you money, once you’re generating revenue. Standards Nonprofits have significantly weaker feedback mechanisms compared to for-profits. They are often difficult to evaluate and lack a natural kill function. Few people are going to complain that you provided bad service when it didn’t cost them anything. Most nonprofits are not very ambitious, despite having large moral ambitions. It’s challenging to find talented people willing to accept a substantial pay cut to work with you. For-profits are considerably more likely to create something that people actually want. Learning Potential Most people should be trying to put themselves in a better position to do useful work later on. People often report learning a great deal from working at high-growth companies, building interesting connection
 ·  · 31m read
 · 
James Özden and Sam Glover at Social Change Lab wrote a literature review on protest outcomes[1] as part of a broader investigation[2] on protest effectiveness. The report covers multiple lines of evidence and addresses many relevant questions, but does not say much about the methodological quality of the research. So that's what I'm going to do today. I reviewed the evidence on protest outcomes, focusing only on the highest-quality research, to answer two questions: 1. Do protests work? 2. Are Social Change Lab's conclusions consistent with the highest-quality evidence? Here's what I found: Do protests work? Highly likely (credence: 90%) in certain contexts, although it's unclear how well the results generalize. [More] Are Social Change Lab's conclusions consistent with the highest-quality evidence? Yes—the report's core claims are well-supported, although it overstates the strength of some of the evidence. [More] Cross-posted from my website. Introduction This article serves two purposes: First, it analyzes the evidence on protest outcomes. Second, it critically reviews the Social Change Lab literature review. Social Change Lab is not the only group that has reviewed protest effectiveness. I was able to find four literature reviews: 1. Animal Charity Evaluators (2018), Protest Intervention Report. 2. Orazani et al. (2021), Social movement strategy (nonviolent vs. violent) and the garnering of third-party support: A meta-analysis. 3. Social Change Lab – Ozden & Glover (2022), Literature Review: Protest Outcomes. 4. Shuman et al. (2024), When Are Social Protests Effective? The Animal Charity Evaluators review did not include many studies, and did not cite any natural experiments (only one had been published as of 2018). Orazani et al. (2021)[3] is a nice meta-analysis—it finds that when you show people news articles about nonviolent protests, they are more likely to express support for the protesters' cause. But what people say in a lab setting mig