ETA: Per kave's comment on LessWrong, this project might not actually have happened as described.
The anonymous review of The Anti-Politics Machine published on Astral Codex X focuses on a case study of a World Bank intervention in Lesotho, and tells a story about it:
The World Bank staff drew reasonable-seeming conclusions from sparse data, and made well-intentioned recommendations on that basis. However, the recommended programs failed, due to factors that would have been revealed by a careful historical and ethnographic investigation of the area in question. Therefore, we should spend more resources engaging in such investigations in order to make better-informed World Bank style resource allocation decisions. So goes the story.
It seems to me that the World Bank recommendations were not the natural ones an honest well-intentioned person would have made with the information at hand. Instead they are heavily biased towards top-down authoritarian schemes, due to a combination of perverse incentives, procedures that separate data-gathering from implementation, and an ideology that makes this seem like the natural and normal thing to do.
Ideology
Within an evidential framework such as Bayesianism, statistics are a specific type of evidence, drawn from low-dimensional quantitative data, with many more observations than degrees of freedom, collected by an automated process decoupled from the process that uses the evidence to decide and act. Within this framework, the parts of the effective altruist narrative related to global poverty might seem to be claiming that, while of course you can help others somewhat by acting locally, statistics allows us to identify opportunities to do much more good by acting on people very distant from us, because we have much more purchasing power than they do (the implied thesis of GiveDirectly), or have better access to information (the implied thesis of every EA global poverty charity except GiveDirectly).
The review begins by affirming an ideology within which the idea of evidence has been not augmented but replaced by the idea of statistics:
If you want your charitable giving to mean something, you also need to measure your favorite program’s effects with good statistical data.
If only statistics are meaningful, then you do not meaningfully understand the material conditions of your life, your sensorium is not meaningful, you cannot help an individual known to you by using your understanding of your own circumstances, and the only information with meaning is the information endorsed by a mysterious-to-itself process by which a large data collection and interpretation agency such as a modern state socially constructs an opinion using statistical methods. Of course such a position rules out as a meaningful intervention not only feeding a hungry person in front of you, but also long-run AI safety work, since while the former case has too many degrees of freedom and too small a sample size to make statistical inferences, in the latter case the relevant statistics could only possibly be collected after the program decisively succeeded.
Problem
The case study begins with three summary facts known to the World Bank staff making recommendations:
- Most of the population in rural Lesotho grew crops, but they did not make very much income from them.
- More than 60% of the area’s young men were working in mines in nearby South Africa and sending back remittances.
- Many families had large flocks of underfed cattle. Even when money was tight, the team rarely observed cattle sales.
The idea is that these facts are true-but-misleading, and a much more extensive up-front ethnographic and historical investigation would be required to act constructively.
As an exercise, I thought about what I might recommend in a situation where all I knew was those three facts.
Most of the population in rural Lesotho grew crops, but they did not make very much income from them.
Conspicuously absent from this is an estimate of how much land the population possesses, and its agricultural potential. The raw acreage per capita can be estimated from population numbers, and one could look at the agricultural yields (and thus revenue) of similar terrain elsewhere. Their actual crop yields can then be compared to the income figures to see whether the problem, if any, is yields or pricing.
If it seems like Lesothans are growing crops just fine, but collecting below-market prices, then they might benefit from better access to global markets via roads or other transportation links, or better information about global markets via telecommunications. It's easy to check if they have cell phones and right-of-way to a nearby large market.
If on the other hand the land seems underexploited, that suggests insufficient access to capital. This could be solved in either of two ways: an outside investor could profitably lend the Lesothans money to invest in agricultural equipment and supplies to improve their yields, or if the Lesothans lack the skills or time to manage that project themselves, they could rent out their land to others willing and able to do so, providing them with direct revenue.
If any of these business opportunities were viable, of course, there would need to be some reason why it hadn't already been exploited. One reason could simply be that no one with access to capital or global markets had put the work into understanding the Lesothans' situation, i.e. that the World Bank has an information advantage it can exploit to broker a deal that otherwise could not happen.
Another reason the deal hasn't happened yet could be that investors are wary of political risk. The World Bank might enable a deal by insuring the investor against expropriation, but if the state's inclined to expropriate from anyone with a visible surplus, then it's not clear that one would be doing the Lesothans a favor by legibly enriching them. The main way I could see an institution like the World Bank being helpful is if they have the leverage to prevent such an expropriation, and therefore collect a profit on the insurance they sold.
More than 60% of the area’s young men were working in mines in nearby South Africa and sending back remittances.
When sending remittances, how much do they lose in fraud or payment processing fees? If a lot, setting up a vouched-for honest intermediary could help. Likewise, do they have access to convenient, cheap transportation?
Another possible problem could be if the young men are capturing only a very small amount of the surplus produced by their labor; if so, helping them bargain with their employers collectively might allow them to earn more.
Many families had large flocks of underfed cattle. Even when money was tight, the team rarely observed cattle sales.
According to what metric are the cattle underfed? Are they yielding less milk, meat, or offspring than they otherwise would under economically optimal feeding? If so, this suggests a profitable investment scheme in which an outside investor either lends the Lesothans the money to feed their cattle adequately, or buys or rents the cattle, feeds them optimally, and gets more out of them than the Lesothans otherwise would.
The above ideas all involve either directly proposing a deal or a specific proposal for further investigation into the Lesothans' circumstances.
The review tells us that the development economists recommended programs that "failed" based on inadequate information, but before actually telling us what they recommended, spends several paragraphs on vague litanies such as:
The World Bank report’s fundamental misdiagnosis of the challenges Lesotho faced formed the basis for a series of failed “development initiatives”, most notably the Thaba-Tseka Development Project, a joint venture funded by the Canadian International Development Agency, the World Bank, the Government of Lesotho, and the UK Overseas Development Ministry.
Finally, a few paragraphs into the second major section of the review, we can read a concrete description of some things that were tried:
the best plots of land in the village had been forcibly confiscated to make room for wood and pony lots, without any sort of compensation
Diagnosis
Stealing the locals' land to plant trees and raise ponies is a totally bonkers response to the three summary facts enumerated. If the World Bank bureaucrats were aware of the likely concrete implementation of their recommendations, then they were not making a mistake, they were recommending a campaign of centralization of power similar to Stalin's collectivization of agriculture, albeit an incrementalist one. There may be valid reasons to something like that, e.g. the state might need to extract more resources for use elsewhere, but helping the locals directly affected is not one of them.
If, on the other hand, the system was set up to conceal the implementation details from the World Bank, that would seem to be the root problem - and that is also not the sort of thing that happens purely by mistake.
It's also not a mistake that there seems to have been little overlap between the kinds of ideas I proposed - ideas that respect the autonomy of the people involved, ideas that would occur to anyone who understood the content of introductory college-level courses in microeconomics and finance, ideas that would have occurred immediately to anyone who understood the standard content of an MBA, ideas that I regularly read about implementations of in the pages of The Economist in the '00s - and, on the other hand, the ideas that the World Bank team proposed.
I'm having some difficulty pinning down what the reviewer's diagnosis is, but initially it doesn't seem like they disagree. In the first section, I read:
But even more seriously, the project was so enveloped in “development discourse” that nobody thought to question whether they were working on problems their “recipients” cared about, or merely the ones the “tools of development” were capable of solving. As Ferguson writes, “The promise that crop farming could be revolutionized through the application of a well-known package of technical inputs was so firmly written into the project’s design that it was difficult for those on the scene to challenge it, or even to confront it.”
[...]
Part of this, perhaps, comes from the usual overconfidence that other social scientists like to accuse economists of. But there are much bigger systemic problems at play. Development work tends to run on short timelines: grad students and postdocs need to publish quickly for their careers to advance, NGO funding runs on 5-ish year cycles, and charities (particularly in “high-risk” areas) face extremely high employee turnover rates. This simultaneously limits the accumulation of institutional knowledge, while incentivizing practitioners away from the time-intensive process of understanding a particular context in favor of “getting results quick.”
Note that if "results" meant "benefit to the locals," the recommendations would not be the best strategy for "getting results quick" - that would be allocating the development budget to cash transfers to the Lesothans being "helped," which seems like the sort of thing that might be done within a week and could likely be done within a month. In context, "getting results quick" means quickly justifying a project, i.e. a job creation scheme, aka a boondoggle.
Later, in the second section:
Two things stand out to me from this story. First, the “development discourse” lens served to focus the practitioners’ attention on a handful of technical variables (quantity of wood, quality of pony), and kept them from thinking about any repercussions they hadn’t thought to measure.
This is a serious problem, because “negative effects on things that aren’t your primary outcome” are pretty common in the development literature. High-paying medical NGOs can pull talent away from government jobs. Foreign aid can worsen ongoing conflicts. Unconditional cash transfers can hurt neighbors who didn’t receive the cash. And the literature we have is implicitly conditioned on “only examining the variables academics have thought to look at” -- surely our tools have rendered other effects completely invisible!
Second, the project organizers somewhat naively ignored the political goals of the government they’d partnered with, and therefore the extent to which these goals were shaping the project.
This would seem to suggest that the problem is that the World Bank is committed - both through its institutional practices and ideology - to implementing a class of frequently destructive policies, and using statistical evidence to justify the set of actions they already have in mind. This is not a defective form of, but an alternative to, reasoning about the situation implied by their statistical summary, forming specific hypotheses about how to help the locals, and then investigating whether the hypotheses are workable. (First-principles reasoning about cash transfers would immediately identify negative spillover from inflation as a concern, though I don't see how someone would expect that to be a net concern.)
The institutional commitments are similar to the ones described in Moral Mazes - in particular the 5-year cycles remind me of the practice of "milking" a division by deferring maintenance, which makes short-run financial numbers look better, under the assumption that you will be promoted or transferred before anything too bad happens due to neglected maintenance, so you won't be held accountable. Parkinson's Law is even more relevant. I expect that anyone drafting a World Bank recommendation has to follow these rules:
- You mustn't recommend something that would reduce the number of people under your boss's authority.
- You mustn't draw any conclusions that would invalidate an important premise of the World Bank's justification for existence, or your department's.
- You must recommend something that involves the disbursement of funds through a limited set of official structures to do things for the locals.
- Recommendations for further investigation may be used to justify the drafting of another World Bank report, or an expensive formalized RCT or survey, but not someone just going around looking and asking questions.
Such constraints are generally not consciously thought of as restrictions on a larger set of natural possibilities, but instead internalized as limits on which actions are thinkable in the first place. This is part of what makes institutional reform difficult.
Recommendation
The natural conclusion here would simply be to discredit and defund institutions similar to the World Bank relative to other things someone might do to help others, like thinking carefully about decision theory or asking a stranger in distress what sort of help they need. But the reviewer instead proposes funding a larger data-gathering bureaucracy, employing a greater number of experts from a wider variety of fields, to form a more detailed initial picture of local situations, to be fed into the same broken bureaucracy.
I'd like to see a cost-benefit analysis.
Your post seems to perfectly underline the classic aid critique of drawing too many conclusions from fitting too few facts, but perhaps not in the way you intended
As the original review linked notes, Ferguson's Anti Politics Machine argues that development economists working on Lesotho in the World Bank drew conclusions from basic statistics and economic theory which they would never have made if they had a wider understanding of Lesotho culture or had at least asked more Lesothans what they wanted. A key example detailed is that Lesothans rarely sold their cattle, which World Bank economists concluded was due to lack of market access, but Ferguson argues is a complete misunderstanding of how Lesothans saw cattle herds (bred as inherited wealth, not a source of income from selling off).
For some reason rather than engage with this detail, you choose to derive your own conclusions from a summary of the same basic facts the World Bank economists based their recommendations on, which happen to include the same conclusions the World Bank economists jumped to: if there's barely any cattle sales it's probably a case of needing better access to markets. As you correctly note, this is the sort of analysis people who have completed an introductory microeconomics course (i.e. basically everyone carrying out economic research at the World Bank...) are likely to conduct.
Similarly, the Lesotho government's privatization of select plots of land that outraged villagers in a culture where land was mostly commonly owned[1] sounds a lot less like something Stalin would approve of and a lot more like a dubious idea that gets signed off by someone who's very familiar with microeconomic theory of land improvement and has found the evidence of underexploitation of land for crops you propose looking for, but hasn't considered that the Lesothan government and local people might have other motivations
I'm agnostic about the benefits of the sort of full blown "historical and ethnographic study", that Ferguson recommends, but it sure beats deriving the ills of complex programs and organizations from a handful of bullet points.
in fairness, this detail isn't in the original article, though it's not exactly an obscure detail you wouldn't expect anyone positing land reform as the real problem to have found out...
My understanding of the term “privatization” is that it generally refers to the voluntary sale of state assets, by the state. That doesn’t seem like quite the same thing as the state expropriating and possibly selling assets that were previously understood to be owned and administered by some smaller community within the state. Am I missing some important detail here?
Individuals within communities didn't own the land, there were customary rights to use it as a commons, generally apportioned at the whim of a local chief. The Lesothan government's proposals were [at least superficially] compatible with orthodox microeconomic theory that the land would yield more if portions of it were fenced off and intensively farmed by land-owning individuals or corporations, and to the limited extend that land titles actually existed in Lesotho, the government and their favoured chiefs were entitled to put fences around valuable portions and develop, lease or sell it to people with the means to exploit it. Lesothans who customarily grazed cattle or collected reeds from those portions of land before nominal landowners fenced it off obviously felt differently.
The process probably resembled the enclosure of common land in 18th century Great Britain more than privatization of state-owned industries (also commonly recommended by the World Bank), but it certainly had far more to do with putting land into private ownership than Stalinist collectivization. There's not much doubt that the government of the time was authoritarian and that its process for allocating land was corrupt, but it was fundamentally driven by the market logic that privately owned land would see more capital investment and higher yields. Ferguson wasn't accusing the World Bank of paying too little attention to "ideas that would occur to anyone who understood the content of introductory college-level courses in microeconomics and finance", he was accusing them of not understanding anything else.
Enclosure acts seem like the correct analogy. And I'd say the enclosure acts and 20th century Soviet modernization were along some relevant dimensions more similar to each other than either is to a decentralization of economic decisionmaking.
The distinction I'm trying to draw attention to in this post is one between unironically believing microeconomics and modern academic finance as descriptive theories that help one interpret the environment in which one lives and has real embedded experience of - treating them as stage 1 simulacra - and, on the other hand, treating those theories as stage 3 simulacra, a bad-faith substitute for interpreting one's environment that serves to entitle one's identity to false credit and the concomitant extraction of resources from less entitled groups. The former attitude would reason from evidence of inefficiency, to predictions about profitable deals one could strike with the locals. The latter would lead to the sort of thing that actually happened. This us approximately the difference between liberalism and neoliberalism.
I don’t think I stated or drew this conclusion. You might be confusing it with the bit about crop sales.
You wrote about "this suggests a profitable investment scheme in which an outside investor either lends the Lesothans the money to feed their cattle adequately, or buys or rents the cattle, feeds them optimally, and gets more out of them than the Lesothans otherwise would" which sounds a lot like trying to create markets to me. The World Bank also tried creating profitable markets for cattle and cattle field and educating farmers to get more out of them than they otherwise would. According to Ferguson, this was a mistake because the Lesothans had no desire to market their cattle, and much more realistic routes to boost their income than trying to maximize milk yields.
(he also argued the same thing about crop sales; it was supplementary food they had little intention of selling)
Executive summary: The World Bank's failed intervention in Lesotho reveals how bureaucratic constraints and statistical ideology can lead to harmful top-down programs, rather than simple market-based or direct aid solutions that would better serve local populations.
Key points:
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.