As we continue to grow, GiveWell seeks to maximize both the cost-effectiveness of the funding we direct and the likely room for more funding of the programs we support. We think we’ve identified a category of interventions that rates really well on both: water treatment, such as chlorination.
This is a major update for us. Before 2020, based on the available evidence, we didn’t believe that water quality interventions had a large enough effect on mortality to make them a competitive target for funding. We’ve since seen new evidence that has led us to significantly increase our estimate of the mortality reduction in young children that’s attributable to these interventions: a 14% reduction in mortality from any cause, up from around 3%.
Though we have remaining uncertainties about these numbers, we’ve substantially updated our view of the promisingness of water treatment. Where we previously found that Evidence Action’s Dispensers for Safe Water program was about as cost-effective as unconditional cash transfers, we now believe it’s about four to eight times as cost-effective, depending on the location. That was a primary factor in our decision to recommend a grant of up to $64.7 million to Dispensers for Safe Water in January 2022.
We’re sharing this news in brief form before we’ve published a grant page, because we’re excited about the potential of this grant and what it represents. It’s an area of work we haven’t supported to a significant degree in the past, but one that we now think could absorb hundreds of millions of dollars in funding for cost-effective programming.
The problem and the intervention
In low-income settings, contaminated water is a major cause of diarrhea, a leading cause of death in children under five years old. Several interventions exist to either purify water or protect it from contamination in the first place, but chlorination has a number of features that made it an attractive intervention for us to explore: it is inexpensive and widely used, several charities are already set up to implement it, and the technology behind it is well established.
Chlorine is well-known as a disinfectant; it reacts with disease-causing microorganisms in water, inactivating viruses and bacteria. There is evidence that chlorination programs, such as distributing chlorine to households, reduce diarrhea in children, but there had been scant evidence that such interventions reduce mortality, which is the single largest driver of cost-effectiveness in our models of water quality interventions. Before 2020, we estimated the effect chlorination had on the prevalence of diarrhea, and then extrapolated from that effect to arrive at an indirect mortality reduction estimate of roughly 3%. This translated to a low cost-effectiveness estimate relative to the programs GiveWell typically funds.
What led us to update
In mid-2020, Michael Kremer, one of the researchers who conducted the initial trial studying dispensers for safe water, and now a researcher at the University of Chicago who has extensively studied global health interventions, shared with us preliminary results from his team’s new meta-analysis of mortality data from randomized controlled trials (RCTs) of water quality interventions. (The working paper has since been published, here.) Pooling studies allowed Kremer’s team to obtain statistically significant results from studies that weren’t individually large enough to produce such results on their own. These results suggest that water treatments have a significant effect on mortality from any cause (“all-cause mortality”) in children under five—Kremer et al. currently estimate a roughly 25% reduction.
Their finding is especially noteworthy because it comes from an analysis of direct evidence on mortality, and it is eight times larger than an indirect estimate that extrapolates from diarrhea morbidity. This implies that water treatment yields a mortality benefit that cannot be fully explained by reduction in diarrhea alone.
This led us to take a deeper look at the evidence and revisit our cost-effectiveness analysis. Using the new data as a foundation, we completed our own meta-analysis of RCTs of chlorination interventions, arriving at a mortality reduction estimate in children under five of about 14%. After adjustments, which were informed by (among other things) local diarrhea mortality rates and differences between the interventions used in the trials versus in the charity context, we estimate that two types of chlorination interventions—Dispensers for Safe Water and in-line chlorination (both described below)—reduce all-cause mortality in under-five children by 6 to 11%, depending on the program and location. This increase in mortality reduction, combined with the additional benefits modeled in our current analysis, corresponded with an increase in cost-effectiveness, which made these programs look much more promising.
In a future blog post, we’ll explain more about what led us to complete our own analysis of the mortality data. More on the process we undertook to arrive at our estimate is in our water quality intervention report.
The program
Dispensers for Safe Water installs, maintains, and promotes use of chlorine dispensers at rural or remote water collection sites in Uganda, Malawi, and Kenya. Our 2018 report on Dispensers for Safe Water concluded that, due to our uncertainty over mortality effects at the time and a model that found Dispensers for Safe Water to be about as cost-effective as unconditional cash transfers, it didn’t meet our bar for directing funding.
When we revisited Dispensers for Safe Water after our re-analysis of mortality effects, it began to look much more cost-effective—between four and eight times as cost-effective as unconditional cash transfers, depending on location. That new estimate, combined with our confidence in Evidence Action as an implementer and our understanding from conversations with Evidence Action and others that water treatment is a particularly neglected focus area, led us to recommend a grant to Dispensers for Safe Water of up to $64.7 million over a period of seven years.
Longer-term, we think there’s a true funding gap for this work that’s unlikely to be filled by other funders. We speculatively estimate that Dispensers for Safe Water, or programs targeting similar populations at a comparable level of cost-effectiveness, could cost-effectively use $350 million per year globally. We’ll publish more details in our forthcoming write-up of the grant.
Other water quality interventions
Our current view of water treatment’s effectiveness at reducing mortality and its relative neglectedness has led us to begin exploring water quality interventions besides Dispensers for Safe Water. We’ve reached a fairly advanced stage in our investigation of one such intervention, Evidence Action’s in-line chlorination program, a technology that automatically disinfects the water in communal water tanks with chlorine rather than requiring an individual user to add it. We’re in much earlier stages of researching other programs, such as distributing vouchers for at-home chlorination supplies, distributing water filters, and protecting natural springs from contamination.
In conclusion
We’re excited about the potential of this grant to Dispensers for Safe Water to maintain and expand on a much-needed, life-saving service, and we’re hopeful that water treatment generally will provide even more opportunities for us to do good with our growing resources. Our research is far from done—due to our remaining uncertainties, we want to fund additional studies on the effects of water treatment to help us refine our mortality reduction estimate and pressure-test our current findings. This is an integral part of GiveWell’s commitment to truth-seeking—seeking out new data and being open to revising our views.
For more on our investigation, see our intervention report on water quality. More details on the Dispensers for Safe Water grant will be available in a forthcoming grant page.
Thanks for all your hard work! This is very exciting. Do you feel that the mortality gap relative diarrhea is something in need of explanation? It increase my confidence in the results of the meta-analysis if there was some indication where it was coming from. Or would something like waterborne parasites, malarial mosquito larvae in the water tanks be enough to cover it?
Hi, Joel,
This is a big question, and one we'd very much like to have a better explanation for! It's important to note that our estimates are uncertain, so the gap between the mortality benefit of water quality interventions and mortality attributed to diarrhea is not necessarily as large as it appears. But we do find it plausible that a larger-than-expected mortality reduction would exist, for a few reasons:
For more details and citations for the above, see this section of our water quality intervention report (particularly beginning with "We are uncertain about our central estimate of the mortality impact of chlorination…").
In the course of our investigation, we also spoke to several researchers who found it plausible that water quality improvements would have a larger-than-expected mortality benefit. These researchers offered their own insight into why water treatment might also reduce non–diarrheal illness, or why the reduction in diarrhea mortality might be larger than what one would calculate indirectly from diarrhea morbidity. See our notes from these conversations here, here, and here.
In sum, we still have uncertainty about what explains the gap between mortality reduction estimates, but the impact of averting waterborne diseases on death from non-waterborne diseases probably plays a role. We think that further empirical evidence of the size of the mortality effect, or a better understanding of the mechanisms that link water quality to mortality, could help to substantially reduce our uncertainty.
Thanks you very much for taking the time to reply!