Persistence - A critical review [ABRIDGED]

by Jsevillamol3 min read10th Nov 20216 comments

82

Cultural persistenceHistoryEconomics
Frontpage

[this is a link post to the preprint Persistence - a Critical Review, by Jaime Sevilla]

EDIT: For a more informal and less dry piece read my takeaways here

IN SHORT: I review, replicate and extend the analysis from seven econometric papers studying how events that happened to and values held by our ancestors affect their descendants several generations afterwards (intergenerational persistence). I argue that together the papers provide moderate evidence of the existence of long term causal effects mediated by parentage.

KEYWORDS: persistence, cultural persistence, economic history, multiple hypothesis testing, post design power analysis, spatial autocorrelation bias, causality, natural experiments, instrumental variables.

Intergenerational persistence is an important topic for Effective Altruism, because it can help us understand how our actions today can affect many generations after. I undertook this research to help us shed light on whether cultural interventions (like increasing the degree at which present people value truth and cooperation) can be an effective way of affecting the long-term future.

The papers I review are:

HIGHLIGHTS:

  • I discuss a gold standard for cultural persistence studies, covering how to (1) identify robust long term correlations via regression studies under different sets of controls, (2) identify causal effects via natural experiments and (3) identify whether culture is a significant mediator via children-of-immigrant studies. More
  • I find that many of the papers manage to find statistically significant results. A naive aggregation of the estimated correlation effect sizes suggests that future correlational studies might find effects of around β ≈ 0.28 (0.13) standard deviations per standard deviation of exposure variation. That is, future studies in similar topics should expect to find that one standard deviation of variation on an event would predict ~28% of variation in long term outcomes. However it is hard to rule out spurious correlations due to issues such as spatial autocorrelation or outliers. More
  • Some of the papers attempt to study causation via natural experiments. While a couple of such papers arguably succeed in identifying a causal effect, we cannot discard that subsequent robustness checks will cast doubt on the results. A naive aggregation of the estimated correlation effect sizes suggests that future causal studies might find effects of around β ≈ 0.11 (0.02) standard deviations per standard deviation of exposure variation. That is, future studies in similar topics should expect to find that one standard deviation of difference on an event would cause ~11% of variation in long term outcomes. More
  • I find that children-of-immigrant analysis suggests the possibility of long term persistence of variation mediated by parentage. The authors of the papers tend to explain this persistence in terms of cultural variation, relying mostly on historical accounts as evidence. More
  • Whether long term persistence of variation usually stays constant, wanes or increases with time is an open question. Studying better these dynamics of persistence would be critical to understand the very long-term impact of cultural interventions today. More

 

Table summary of the papers I reviewed. More

Acknowledgements

This work was funded by the Forethought Foundation, and helped inform the decision of whether to rely on results about cultural persistence in William MacAskill’s forthcoming book.

Thanks to David Rhys Bernard, Nathan Nunn, Aron Valinder, Luisa Rodriguez, William MacAskill, Florian Penner, Ehud Reiter, Faatima Osman, Pablo Villalobos, Ronja Lutz and Neel Nanda for feedback and constructive criticism.

Leticia García provided invaluable research assistance, including carefully reviewing all the analyses conducted in the paper. All remaining mistakes are solely my fault.

Pablo Villalobos, Ronja Lutz and Neel Nanda assisted me in acquiring and cleaning the data I needed for the analyses, for which I am incredibly grateful.

David Roodman and Julian Jamison provided a blind peer review of this paper.

Nuño Sempere provided comments on this post.

I welcome comments and feedback on the piece. Read the rest of the piece.

82

6 comments, sorted by Highlighting new comments since Today at 7:04 PM
New Comment

I'm really excited to see this and look into it. I'm working on some long-term persistence issues, and this is largely in line with my intuitive feel for the literature. I haven't looked at the Church-WEIRDness one, though, and now I'm eager to read that one.

I think it's really cool that you did this. It's been on my to do list to look into some persistence studies but I've not got round to it and this seems like a really helpful analysis.

How did you select the papers that you selected to review? E.g. was it due to their focus, their methodology, how well cited they were, something else, or nothing in particular? (For context I have no sense of how many papers using roughly similar methodology there are, so for all I know this could be all of them! I skimmed the preprint and didn't see a mention of this, but could have just missed it.)

Thank you!

 

These papers were ones that William MacAskill was considering citing in his forthcoming book. FF hired me to thoroughly check them.

There is definitely many other persistence papers I didn't cover!

Eg:

  • Acemoglu et al, Colonial Origins
  • Acemoglu et al, Reversal of Fortune
  • Woodberry (2012). The Missionary Roots of Liberal Democracy
  • All the papers cited in Kelly's Understanding Persistence

And many others.

You might also find Arroyo-Abad and Maurer's review of the persistence literature interesting.

EDIT: Faatima Osman questioned whether it was fair to exclude respondents from Benin, Ghana and Nigeria in Nunn and Wantchekon's paper, given that Nigeria is the most populated country in Africa by far.

And in hindsight I think she is totally right - respondents from these countries are ~25% of the sample! I now believe that its unfair to call these respondents outliers. Correspondingly, my trust in Nunn and Wantchekon's paper has gone up, since Kelly's critique was my main concern about it.

I did a quick search for any citations of Greg Clark, and didn't see any. There are other economic historians like him who have found not merely persistence of the status quo but robustness to disruptions to it.