Hide table of contents

[Edit: I think readers should look at Stephen's comment below, which points out key weaknesses of the article and reasons to doubt its conclusions.]

Sigal Samuel from Vox's Future Perfect recently published an article I really appreciated, both because I learned some new things and because it seriously discusses important questions and overcomes the lack of newsworthiness[1] of long-running studies: "What’s the best way to help extremely poor people? After 20 years, the evidence is in." (Subtitle: Is it really useful to “teach a person to fish” or should you just give them the damn fish already?

The article discusses evidence for the effectiveness of cash transfer programs and how that compares to "ultra-poor graduation programs," where 

the idea [is] that offering a combo of assets and training and cash — instead of just, say, cash — can trigger a virtuous cycle that ultimately helps ultra-poor people escape poverty. For example, you can offer people livestock plus training on how to make money off that livestock plus a bit of cash to sustain them while they get things up and running.

Some takeaways

I'm jotting down some takeaways, with the caveats that I threw this link-post together really fast, so this could easily contain errors or misunderstandings (I'd appreciate corrections in the comments!), and that I haven't checked the cited studies myself — I'm just summarizing some parts of the Vox article.

  • The evidence on graduation programs is very good, even after 10 years (other programs will sometimes seem very promising for the first few years, but less promising after a while, in part because control groups catch up)
  • But it's possible that the potential of the programs is over-stated because the existing trials were with some of the people who were easiest to help in such a way, so this approach is harder to generalize. Similar programs in other places may fail because the local markets are too unstable, there aren't easily identifiable appropriate groups of people to work with on this, or the programs themselves are too expensive (e.g. if it's more expensive to pay local residents to run the program).
  • In general, there isn't enough understanding of when one program is better than another, and these things do vary a lot.
  • It's also possible that cash transfer programs, especially one's like GiveDirectly's, "geographically saturate"[2] an area in a way that has spillover effects which, notably, could make control groups catch up to cash transfer recipients over time (making the program seem worse than it actually is).
  • "When you talk to people in the pro-graduation camp and people in the pro-cash camp, you start to realize a funny thing: These two camps are actually moving closer and closer together over time. The gap between 'teach a man to fish' and 'give a man a fish' is narrowing." There are in-between options that might (sometimes) be better than both simple cash transfer programs and full graduation programs; cash transfers along with education might be a promising version of a "minimum viable" graduation program — and something that's easier to scale.

I'd be interested in other people's takes on the article, and I've pasted some excerpts below. 

Excerpts

The intro: 

If you want to fight poverty, you probably intuitively feel that the worst-off people are the ones who should be prioritized. As difficult as it is to live on a few bucks a day, someone who’s living on just $1.90 a day clearly has it worse, and it makes sense to think you should try extra hard to help the poorest of the poor.

It’s a big moral problem, then, that a lot of anti-poverty programs fail to successfully do that.

That problem has bothered BRAC, a major international development charity based in Bangladesh, since the 1990s. Back then, the charity was working on voguish anti-poverty programs. Microfinance was all the rage, but it was becoming clear that microloans weren’t reaching the poorest households. Nobody wanted to lend to them because who knew if they could pay back the loan? And the poorest households often didn’t want to borrow because they weren’t confident that they could figure out how to turn a profit and repay.

[...]

The BRAC team decided they needed to try something new if they wanted to lastingly improve life for the worst-off — the “ultra-poor,” as they put it. So in the early 2000s, they went into village after village in Bangladesh, deliberately looked for the poorest people, and talked to them. And what they realized was that the ultra-poor aren’t only poor in terms of cash — they also lack knowledge about how to invest cash, lack confidence in themselves, and lack social ties to the broader community.

“We started realizing that it’s not going to be a simple sort of solution,” Abed said. “It’s going to have to be a package of things, because it has to address multiple vulnerabilities. So then there was this idea of a ‘big push’ investment.”

That “big push” is the idea that offering a combo of assets and training and cash — instead of just, say, cash — can trigger a virtuous cycle that ultimately helps ultra-poor people escape poverty. For example, you can offer people livestock plus training on how to make money off that livestock plus a bit of cash to sustain them while they get things up and running. This premise became the bedrock of what BRAC called the “ultra-poor graduation program,” which aims to “graduate” recipients out of extreme poverty.

On studies over time: 

Sometimes an anti-poverty experiment will show promising results after a year or, say, four years — but by year nine, the results tend to look much less rosy, perhaps because the control group catches up with the treatment group. So it’s very helpful when economists do a 10-year follow-up to check whether the initial results persisted over time.

That’s why it’s worth drilling down into another randomized controlled study published by Duflo, Banerjee, and co-author Garima Sharma just a few months ago. This study followed up on an experiment conducted in West Bengal, India. Ultra-poor people were given two cows or two goats, together with training on how to generate income from the livestock and a small subsistence stipend to keep them going. The researchers found that the initial results persisted, with study subjects enjoying higher income and consumption even a full decade later.

Things are complicated: 

When [Shameran Abed, the executive director of BRAC] looks at all this evidence, he thinks the upshot is clear. “Graduation programs are much more impactful in the longer term,” he said. “I know that for the ultra-poor.”

But others, like Banerjee, are more circumspect. “I don’t think we can say that yet,” he told me. “I think it’s hard to read the evidence.”

Why? For one thing, while graduation programs appear to work great in some places, they’re dependent on the market — and they can run into problems in places where the market is either too dysfunctional or, ironically, too functional.

One randomized trial in India, published in 2012, is an example of the latter. It found that a graduation program yielded no net impact. Although it shifted participants away from agricultural jobs to other sorts of work, they could’ve earned just as much in their original agricultural jobs. While those original jobs were far from big money-makers, wages for agricultural labor had been improving in India, thanks to programs like the ambitious National Rural Employment Guarantee, so adding in a graduation program didn’t really help.

Dysfunctional markets produce their own obstacles. Abed told me about his experience trying to run a graduation program in Balochistan, an extremely dry, desert-like province in southwestern Pakistan, where participants were taught how to run a small business. One problem: There wasn’t a functional market for the businesses to thrive in. “Once they graduated, there wasn’t much to go to,” said Abed. “And there wasn’t microfinance available. So it was very, very difficult.”

Arguably, this points to an issue with the graduation approach’s reliance on the concept of the “poverty trap.” The idea here is that poverty works like gravity: to help someone escape it, you have to get them above a certain escape velocity. If you don’t give them a big push that gets them above that threshold, they’ll eventually sink back into poverty.

“I think the poverty trap concept is very simplistic,” said Miriam Laker-Oketta, a Uganda-based research director at GiveDirectly, which runs cash transfer programs. “It makes it look like you just need to get this one person out of poverty. But because they’re human beings, they’re all connected. The community is all connected. I think we need to be thinking of poverty in terms of communities rather than individuals. It has to be more systemic.”

An image from the article: "Once she received her initial asset transfer of three pigs, Oretha set into motion a plan to expand her business and climb out of poverty in Bong County, Liberia. | BRAC"
  • ^

    Cold take: many news outlets undervalue more continuous, longer-running-but-important stories, because they're not recent or unexpected enough. You can take your pick of lists of criteria for newsworthiness and see if you think this really tracks the importance of given events or phenomena. 

    Related: "Does the news reflect what we die from?" by Our World in Data. 

  • ^

    Here's a potential source/relevant study — I haven't yet read it, but it looks interesting. 

  • Comments6


    Sorted by Click to highlight new comments since:

    [Epistemic status: Writing quickly about a complicated literature - I think the direction of this critique is right but would love to discuss further in the comments!]

    I know authors don't choose their own headlines, but this one really is a ridiculous overstatement. We can't crown something the "best" way to help extremely poor people after only comparing two interventions (graduation vs. cash). And even in a head-to-head comparison, the available evidence suggests that the characteristics and effects of graduation-style programs vary so much across contexts that we should be extremely cautious about generalizing from a study of one particular program in one particular place.

    At the risk of further beating an already dead horse, I also think it's worth re-contextualizing this article in the context of the growth vs. randomista debate (which, IMO, has not advanced much at all since Hauke and John's post in Jan. 2020. Here again is Pritchett's damning figure comparing the gains from a representative $1,000,000,000 investment in Graduation-type programs to the value of various national growth accelerations:

    Overall I feel pretty disappointed at how un-quantitative the linked Vox article is. Sigal summarizes the results of  the latest BRAC study as "study subjects enjoying higher income and consumption even a full decade later." But this is just saying the program has AN effect. This is trivial. What matters is the size of the effect. Unfortunately, Sci-Hub hasn't indexed the 2021 study the Vox article focuses on. But I've looked at the results of previous graduation studies and haven't been blown away. Banerjee et al. (2016) report of BRAC that "seven years after the asset were first distributed, the monthly consumption of those assigned to treatment is 16 dollars–or 25%–higher [than those in the control group]".

    Of course, 16 extra dollars per month works out to $0.50 per day. I believe in diminishing returns to consumption such that an extra 50 cents per day is very meaningful if it boosts your consumption by 25%. But any claims that Graduation helps people "graduate from" or "escape" poverty assumes a low poverty line, such as the standard $1.25 per day. That is, the average very poor Graduation program participant remains very poor after participating in the program. I think it's misleading to talk about them escaping poverty as Vox and Graduation proponents do: e.g. from the article "[BRAC] aims to “graduate” recipients out of extreme poverty."

    I'm impressed by the thoroughness of Sigal's literature review here but still think it understates the extent to which these programs are controversial. In fact, I basically think we shouldn't generalize at all from studies of one particular Graduation program. Some of these issues are discussed in what I think is a pretty good critical article, Kidd & Athias 2019, which I don't see discussed in the Vox piece.

    I really appreciate this comment, thank you!

    I agree with your disappointment about the lack of any quantitative aspect, and I'm adding the paper you linked to my reading list. 

    I've also been planning on reading selected books and papers from Further reading/References in the Growth and the case against randomista development for a while, but if you have other recommendations, I'd love to hear them. 

    Here's a link to the Banerjee paper for those without institutional access.

    Thanks for the link. I want to emphasize that I think this is a very good paper. The intro especially is well worth reading for its description of the program and poverty trap model.

    Here's a relevant quote; the results aren't much of an update as the absolute treatment effect in terms of per-capita consumption didn't change between years 7 and 10.

    Average household per capita consumption was $1.35 (2018 PPP) at baseline in the control group and $2.90 by year ten. [...] [The treatment group's] per capita consumption is $0.60 per day (0.6 standard deviations) higher than the control group at both years seven and ten, and income is 0.3 standard deviations higher. This temporal pattern of growing effects followed by persistence is consistent with the alleviation of a poverty trap, what BRAC describes as the graduation of treated households. However, it is also consistent with there being persistent effects without actually getting out of a trap: the control households do become less poor over time, and the treatment households are still not very rich by the time the treatment effect stabilizes (although their average consumption is above the World Bank threshold for “moderate poverty”).

    (p. 472, emphasis mine)

    Note that the authors' wording is more cautious and nuanced than the Vox article.

    [comment deleted]1
    0
    0

    This reminds me of a video about teaching the different styles of learning (not really backed up by science). The three options were visual learners, auditory learners , etc? It was basic snake oil stuff, someone made a video comparing the styles, and it showed people doing better when....all of the styles were combined into a multimedia presentation.

    Um, ok? So do both. 

    It's just hard to do a randomized controlled study that way.

    So add a third arm, where they do both and compare. That would be a good study.

    Source: https://www.educationnext.org/stubborn-myth-learning-styles-state-teacher-license-prep-materials-debunked-theory/

    Curated and popular this week
    Garrison
     ·  · 7m read
     · 
    This is the full text of a post from "The Obsolete Newsletter," a Substack that I write about the intersection of capitalism, geopolitics, and artificial intelligence. I’m a freelance journalist and the author of a forthcoming book called Obsolete: Power, Profit, and the Race to build Machine Superintelligence. Consider subscribing to stay up to date with my work. Wow. The Wall Street Journal just reported that, "a consortium of investors led by Elon Musk is offering $97.4 billion to buy the nonprofit that controls OpenAI." Technically, they can't actually do that, so I'm going to assume that Musk is trying to buy all of the nonprofit's assets, which include governing control over OpenAI's for-profit, as well as all the profits above the company's profit caps. OpenAI CEO Sam Altman already tweeted, "no thank you but we will buy twitter for $9.74 billion if you want." (Musk, for his part, replied with just the word: "Swindler.") Even if Altman were willing, it's not clear if this bid could even go through. It can probably best be understood as an attempt to throw a wrench in OpenAI's ongoing plan to restructure fully into a for-profit company. To complete the transition, OpenAI needs to compensate its nonprofit for the fair market value of what it is giving up. In October, The Information reported that OpenAI was planning to give the nonprofit at least 25 percent of the new company, at the time, worth $37.5 billion. But in late January, the Financial Times reported that the nonprofit might only receive around $30 billion, "but a final price is yet to be determined." That's still a lot of money, but many experts I've spoken with think it drastically undervalues what the nonprofit is giving up. Musk has sued to block OpenAI's conversion, arguing that he would be irreparably harmed if it went through. But while Musk's suit seems unlikely to succeed, his latest gambit might significantly drive up the price OpenAI has to pay. (My guess is that Altman will still ma
     ·  · 5m read
     · 
    When we built a calculator to help meat-eaters offset the animal welfare impact of their diet through donations (like carbon offsets), we didn't expect it to become one of our most effective tools for engaging new donors. In this post we explain how it works, why it seems particularly promising for increasing support for farmed animal charities, and what you can do to support this work if you think it’s worthwhile. In the comments I’ll also share our answers to some frequently asked questions and concerns some people have when thinking about the idea of an ‘animal welfare offset’. Background FarmKind is a donation platform whose mission is to support the animal movement by raising funds from the general public for some of the most effective charities working to fix factory farming. When we built our platform, we directionally estimated how much a donation to each of our recommended charities helps animals, to show users.  This also made it possible for us to calculate how much someone would need to donate to do as much good for farmed animals as their diet harms them – like carbon offsetting, but for animal welfare. So we built it. What we didn’t expect was how much something we built as a side project would capture peoples’ imaginations!  What it is and what it isn’t What it is:  * An engaging tool for bringing to life the idea that there are still ways to help farmed animals even if you’re unable/unwilling to go vegetarian/vegan. * A way to help people get a rough sense of how much they might want to give to do an amount of good that’s commensurate with the harm to farmed animals caused by their diet What it isn’t:  * A perfectly accurate crystal ball to determine how much a given individual would need to donate to exactly offset their diet. See the caveats here to understand why you shouldn’t take this (or any other charity impact estimate) literally. All models are wrong but some are useful. * A flashy piece of software (yet!). It was built as
     ·  · 16m read
     · 
    Over the years, I have learned many things that are rarely taught about doing cost-benefit or welfare analysis. Here are a few things that I often end up repeating when I mentor individuals or teams working on these kinds of projects: A Point Estimate is Always Wrong For any purpose other than an example calculation, never use a point estimate. Always do all math in terms of confidence intervals. All inputs should be ranges or probability distributions, and all outputs should be presented as confidence intervals. Do not start with a point estimate and add the uncertainty later. From day one, do everything in ranges. Think in terms of foggy clouds of uncertainty. Imagine yourself shrinking the range of uncertainty as you gather more data. This Google Sheets Template allows you to easily set up Monte Carlo estimations that turn probabilistic inputs into confidence-interval outputs. Use Google Sheets I have experience programming in half a dozen languages, including R. Sometimes they are useful or necessary for certain kinds of data analysis. But I have learned that for almost all cost-benefit analyses, it is best to use Google Sheets, for several reasons. The main one is transparency. A cost-benefit or welfare analysis is a public-facing document, not an academic one. You should not use esoteric tools unless absolutely necessary. Anyone in your society with basic literacy and numeracy should be able to read over and double-check your work. When you are done and ready to publish, you make your Sheet visible to everyone, and add a link to it in your report. Then anyone can see what you did, and effortlessly copy your code to refine and extend it, or just play around with different priors and assumptions. This transparency also helps improve results and correct mistakes as you are doing the work. The more people review your math, the better it will be. The number of people who are willing and able to look over a spreadsheet is orders of magnitude higher than the
    Recent opportunities in Global health & development
    63
    · · 1m read