Lant Pritchett on the futility of "smart buys" in developing-world education

by Aaron Gertler8 min read18th Jul 202114 comments

48

EducationCopenhagen Consensus Center
Frontpage

Why I wrote this post

Education isn't something many people explicitly focus on within EA. But I still think it's good to highlight strong writing on a variety of cause areas for a few reasons:

  • Reading about the real world helps you build better models of it.
  • Reading about difficulties with one cause area might help you anticipate similar difficulties in other areas (some of the discussion below could easily apply to health interventions, institutional decision-making, etc.)
  • Lots of people ask why EA isn't very focused on education (or any number of other causes). It's useful to know why some areas are more promising so that we can explain why we prioritize certain things.

I thought Pritchett's writing below (on primary and secondary education in the developing world) was well-written and reasonable (albeit rough, as it consists of an informal blog post and two early-draft PDFs). I learned a lot about developing-world education; a few new ideas, and lots of building on "things I thought I knew" with actual evidence.

If you're interested in education as a cause area, or just as an "interesting thing to think about", I definitely recommend Pritchett's blog post + short paper. If you like those, the long paper is also worth a skim.

The problem with "smart buys" in education

The Copenhagen Consensus hires economists to write lists of "smart buys" — things a donor could fund that would have some impressive impact for developing countries.

In 2008, they asked Lant Pritchett to make one of these lists for education. He sent back a paper without one, and gave back what he'd been paid. He thought such a list would be futile, given that:

  • Promising education RCTs seldom worked at scale
  • Much discussion of education policy was fundamentally confused — the goals, incentives, and theories of development economists didn't match those of local governments, teachers, or students.
    • Thus, recommending ways for governments to maximize an "education output" through their spending wouldn't make sense, as they clearly weren't looking to do this in the first place.
    • If spending were to have an impact, certain factors would have to be in place — factors that were often rare in the developing world. (For example, reducing school fees wouldn't do much unless schools were actually effective at educating students.)

He draws the following analogy:

Suppose I have just read that spinach and broccoli are “cost-effective” foods in providing high nutritional content at low prices. I am in the grocery store and see a fellow shopper whose cart is loaded with food that is both bad for you and expensive (e.g. sugared breakfast cereals) and nothing really nutritious. I could then go up to her/him and make a “recommendation” and give him/her my empirical evidence grounded “smart buy” advice: “Hello stranger, you should buy some broccoli because it is a cost effective source of vitamins.” 

One can imagine many outcomes from this but perhaps the least plausible response is: “Thanks, fellow stranger, I will now buy some broccoli and integrate this cost-effective source of vitamins into my regular food consumption habits.”

Take the analogy a step further and suppose I have an altruistic interest in the health of my fellow shopper and so I just buy and broccoli and spinach and put it into his/her shopping bags for free. Again, one can imagine many outcomes from this action of mine, but I would think the most probable is that some broccoli and spinach gets thrown away.

This doesn't mean there aren't good ways to invest in better education — just that RCT evidence alone is a poor substitute for understanding incentives and structures on the relevant local/national level, and that those factors will doom some naïve approaches.

In the end, Pritchett wrote two papers for the Copenhagen Consensus:

  • "The Challenge of the Lack of Education" sums up a huge amount of evidence on schooling practices, and ends with a set of potential areas for improvement — no "smart buys" with estimated impact, but some general approaches that might work if certain factors are in place.
  • His perspective paper on education, a reply to "smart buy" recommendations from other economists, explains some key difficulties in trying to "buy" more education. It's much shorter, so I'd recommend starting here if you feel like spending some time in the head of a smart, cantankerous researcher.
     

Excerpts from the short paper

There are very large private returns to schooling—on average, across the 42 developing countries [the economists examine], an additional year of schooling is associated with an 8.2 percent increase in wages for men and a 10.3 percent increase in wages for women, and a 9.2 percent increase in urban areas and 8 percent in rural areas. 

Why does this make [the economists'] life difficult? Because if I want to justify public spending on something, I want the private gains to be small and the public gains to be large. But what they show is that the private gains are considerable, which means that, if one were to justify public spending as a large proportion of the private spending, one needs very large externalities to education. 

 

The most common reason given why children are not attending school is “lack of interest”—which is 47.3 percent of urban children and 44 percent of rural children not in school. 

Why do these children “lack interest”? This is hugely important to understand, as it also influences how we understand the other responses. If someone says the reason a child is not in school is because they “work outside the home” that may just be begging the question as the question is “why do they work outside the home and not go to school?” 

While it is obvious that children not in school work more, it is not so obvious how much is cause (children drop-out of school in order to work) and how much is effect (once children have dropped out (for other reasons) they work more). 

Here is my conjecture. Going to school reveals two things. First, it reveals your adeptness for formal schooling (not some catch all like “intelligence” but just how good at school you are). Second, it reveals the quality of your school. By the time most children reach, say 14 or 15 years old, many of them “lack interest” in schooling because either (a) they have realized they are not adept at schooling (and hence do not like it) or (b) they realize the school they are in is miserable and/or no learning is going on or both (a) and (b). 

Rather than the model of parents pulling children out of school to work (in the market or at home) I would suspect the much more common phenomena is children pleading with their parents to not have to go to school. 

Just as an example, Tyack (1974) tells that Helen Todd, a factory inspector in Chicago interviewed 500 children working in factories (often in dangerous and unpleasant conditions) and asked the question: “If your father had a good job and you didn’t have to work, which would you rather do—go to school or work in a factory?” Of those 500, fully 412 said they would choose factory work. She recounts asking one fourteen-year-old girl in one particularly unpleasant factory (lacquering canes, involving heat and turpentine) why they did not go to school and got the response “School is the fiercest thing you can come up against. Factories ain’t no cinch, but schools is worst.”

 

The results from Progresa—of impacts on attendance but not on measured learning outcomes—is consistent with the view that marginal students were forced back into marginal schools. 

 

According to [the economists], iron supplements to secondary school students has a BCR [benefit-to-cost ratio] of 32. What is the market failure? One can just buy iron supplements on the market right? So if parents knew of these private benefits from a private action, then they should willingly adopt it, right? 

So [instead of] a policy of “paying for iron supplements” why not a policy of “publicizing the benefits of iron supplements”, from which one should get much higher uptake with less cost? 

Similarly with something like Balsakhi tutoring. There is a huge flourishing market in India for private schools and tutoring. If you really believe the BCR is 528—you can get 5,152 in benefits for only 9 dollars—why [not simply make] parents aware of the potentially massive benefits of a particular type of tutoring? The behavioral model is that parents will turn down a low total cost, [hugely beneficial] intervention? 

If not, then spreading the information (the creation of which is a public good) should be enormously more cost effective as a policy than scaling up the program (or alternatively the scaling up of this program should be an enormous cash cow the private sector would willingly scale up).

Excerpt from the long paper

This paper goes into lots of detail on specific interventions in a way that felt hard to excerpt from (the value is in the details!). But I did think this was well said:

Since many of the readers of this document will have no direct experience living or working in the poorer countries of the world in which the lack of education is most pressing I would ask them to abandon now any assumptions based on their personal experience. 

Citizens of the developing world cannot take for granted what every resident of a rich industrial country does. In particular, many of the same words are used in discussions—but they convey problems of different orders of magnitude:

  • When “poor pedagogical approaches” are discussed, it is usually taken for granted the children are not being hit with by the teacher with a stick—not always so.
  • When “under motivated” teachers are discussed, it is usually assumed they at least show up—not always so.
  • When “poor facilities” are discussed, it is usually assumed there is at least a classroom—not always so.
  • When “political patronage” is discussed, it is usually assumed that teaching positions are not openly sold as patronage—not always so.
  • “Low quality teachers” is not assumed to not mean “cannot read and write at a fourth grade level”—not always so.

48

14 comments, sorted by Highlighting new comments since Today at 6:08 AM
New Comment

Quotes from the papers

Even more particularly, government policies that increase the returns to schooling
will be key to raising the demand for education. 

FALSE. See Sri Lanka, Jamaica,  Mauritius, Singapore, South Korea, Taiwan, Indian state of Kerala. All of these  places had increases in schooling before increased economic returns.

Were these increases typically driven by public demand, or driven by top-down government policy? If the latter, Pritchett's point could still stand.

They were driven by government policy, the policy was around changes in  the schooling system + whatever changes were needed to encourage kids to go to school. The changes had NOTHING to do with "increase the returns to schooling" as Pritchett wrongly asserts.

Were these increases typically driven by public demand

This is really hard to tell. If there are no schools in walking distance of your village and hence no one goes to school does it mean there is no demand? If you live in an authoritarian country and know that the dictator will not build schools, and hence no one demands schooling. Is there no demand?

In the case of Kerala, Singapore local governments did all they could to encourage schooling. As a result enrollment increased. Does that mean there was an increase in public demand? (Edit: The governments of Kerala, Singapore also built the schooling system: building, teachers, books etc...)

 

Disentangling government action vs public demand is not so important. There are good practices from Singapore, Kerala etc... that can be learnt by governments the world over.

Now, there are many other ways that spending on primary education can be justified—that education is a universal human right, education is a merit good, the demands of political socialization demand universal education. I suspect that the actual positive theory of education has more to do with those than with the economic returns. But for the purposes of the present exercise of comparing alternative uses of public funds across sectors one cannot invoke “human rights” as a reason to spend on schooling without a counter of “intrinsic values” of an unchanged natural environment, which returns the debate to non-quantifiable values.

I find this (Copenhagen Consensus 2008 Perspective Paper, Pg7) the weakest argument so far, since of course rights have non-infinite moral value and we can extract things like willingness to pay, hedonic utility, etc from how much rights on worth.

I think this is a much more plausible view of much of the drop-out phenomena than is “credit constraints.” First, strictly speaking, “credit constraints” is not a very good description of the problem. Let us take the author’s numbers seriously that the return to schooling is, say, 8-10 percent. Let us suppose that families in developing countries could borrow at the prime interest rate. The real interest rate in many countries in the world is around 8 to 10 percent. So given the opportunity to borrow at prime to finance schooling many households would rationally decline the offer. Second, imagine one relaxes the pure credit constraint—would that money flow into education? Returns on investments from micro-credit programs (which typically have lending rates between 12 and 20 percent per annum) are profitably at those lending rates. One would need more research but the range of investments for which households usually borrow have much higher returns (and quicker) than 8-10 percent.

(Copenhagen Consensus 2008 Perspective Paper, Pg8) 

Huh this take did not survive the test of time well, given the last 13 years of research on microfinance. 

Linch, Can you explain. What did not survive the test of time?

The belief that micro-credit has good investment ROIs for the typical recipient.

I deliberately chose not to use this as one of my chosen excerpts, though I don't think it reveals a weakness in anything Pritchett believes — I read him as a skeptic about these "rights" who nevertheless acknowledges that other people would rather talk about rights than economic return in discussions of education. But whether he believes in the concept or not, your objection to the concept seems correct to me.

Linch we can also use HDI (Human Development Index) and calculate education ~= money

Here is what I get for children's education

6 years schooling = 890 PPP USD per year

9 years schooling = 2800 PPP USD per year

12 years schooling = 8500 PPP USD per year

(I read the non-blockquote parts of the post, skimmed the blockquotes, and did not click through to any of the links.)

It seems like the kind of education discussed in this post is exclusively mass schooling in the developing world, which is not clear from the title or intro section. If that's right, I would suggest editing the title/intro to be clearer about this. The reason is that I am quite interested in improving education so I was interested to read objections to my views, but I tend to focus on technical subjects at the university level so I feel like this post wasn't actually relevant to me.

Fair comment; I've edited the title and the introduction.

Lots of people ask why EA isn't very focused on education (or any number of other causes).

Education is one of 3 parts of Human Development Index, the other being Income and life expectancy.  The fundamental and FATAL problem of the EA community is not giving education equal importance as the other parts of the HDI.

Here I am talking about INTRINSIC value, as an end in itself like life expectancy.

The EA community and Give Well have never published moral weights for education vs money or life expectancy. They have chosen to disregard UNDP and HDI. I wonder is the UNDP part of the outgroup?

 

Try reading the following

"Lots of people ask why EA isn't very focused on education (or any number of other causes)."

"Lots of people ask why EA isn't very focused on life expectancy (or any number of other causes)."

"Lots of people ask why EA isn't very focused on income (or any number of other causes)."

All of the above sound crazy to me, all of them are big buckets and issues. The lack of acknowledgement of the importance of education is the issue.

 

Whether there are "smart buys" in education is a secondary issue. The research might find  cost effective charities working in any combination  of income/life expectancy/education or exclusively in one of them.

The starting point has to be acknowledging the importance of education.

Do you know of any studies showing that people in low-income countries regard their own education as a major source of intrinsic value, apart from its effects on other life outcomes?

I ask because I think most people in the developed world value education primarily because it will help them "succeed in life" (or "get a good job", "move up in the world", etc.). If you gave people in the U.S. a choice between e.g. the experience of being in school for 3 extra years, or an extra $5,000/year in salary, I'd expect almost everyone to choose a higher salary. And I would expect sentiments to be similar in the developing world, if not stronger.

I don't have access to data on this, and am generalizing from how I've seen people behave in my own life and in various nonfiction books and articles I've read. I'd be really curious to see data, and you seem like you might be familiar with relevant literature.

Education is one of many things you can do with your time; I don't see why we'd necessarily privilege it over "spending time with your family", "playing with your friends", or other ways to spend time — apart from its effects on economic welfare, health, and so on.

See GiveWell:

There are a limited number of experimental studies providing direct evidence that education interventions improve the outcomes that we consider most important, such as earnings, health, and rates of marriage and fertility among teenage girls. 

They certainly seem to differ from UNDP if UNDP considers education an intrinsic good, absent other effects on welfare. But I'm willing to bite the bullet and ask whether UNDP is actually right. Is education for its own sake important enough to justify pursuing interventions that provide more education, but less health or money, than other interventions?

Aaron,

   I have written a lot about education on this forum. See this old post of mine

Education is one of many things you can do with your time

I am talking about basic education here (12 years). A child going to school is not losing family time, they are  learning and playing with their friends at school. If there are not at school they might be looking after siblings, grazing the animals, or maybe doing nothing.

See GiveWell:

Givewell's research on education is of really poor quality.   Partly that is because they assume education has no intrinsic value and hence put little effort into it. Partly it's hard to disentangle effects of education because those effects last a lifetime, and can easily be mis-attributed to income, government policy, soap operas, economic policy etc...

regard their own education as a major source of intrinsic value, apart from its effects on other life outcomes?

The question of intrinsic and extrinsic value is not very interesting. We can ask why does health have intrinsic value? Income it is clear has no intrinsic value. UNDP considers Income, education, health to be equally important, because they allow us to leads lives that we want to live i.e. enhance human capability.  See Capability Approach

UNDP considers education an intrinsic good, absent other effects on welfare. But I'm willing to bite the bullet and ask whether UNDP is actually right.

Given the importance of HDI, the research backing HDI (kerala model), the people who created it (Mahbub ul Haq, Amartya Sen +others).  The starting point for any moral weights has to be HDI, we can differ from it but need really solid explanation for substantial deviations from HDI. The burden of proof is on the EA community  and especially Give Well.

After 30 years of HDI, the major changes are Inequality-adjusted Human Development Index (IHDI), Gender Development Index (GDI),  Gender Inequality Index (GII), Multidimensional
Poverty Index (MPI)

In all of those indexes education is given equal importance to health and income. The persistence of education for 30 years in HDI and the newer indexes points to the importance of education.

 

Moral Weights of Education, Income, Health according to UNDP HDI

from http://hdr.undp.org/en/content/hdr-technical-notes

18 years of education = 1 life (85 years)

9 people doubling of income per capita for their entire lives = 1 life

or

765 years of additional income = 1 life

 

Give Well weights are wildly off from this.