Hi Tony, This is Mark again. Thanks for the nudge on this monitoring point!
We agree that real-time monitoring of critical activities is likely to be a cost-effective investment. For example, New Incentives (a program that provides conditional cash transfers for immunizations working with Northern Nigeria health clinics) tracks and shares with us data on cold chain failures, stockouts, rollout delays, security incidents, fraud, double treatment of children, and retention rates (see more information in this intervention report). We think this type of monitoring is a good investment for New Incentives and want others to do something similar, because it:
We're still trying to understand the trade-offs between grantees doing this internally vs. externally, and we expect to learn as we go. It makes sense to me that it is usually most efficient and effective for grantees to do the core of this monitoring internally since we want them to be able to address these issues in real-time (point 1) rather than waiting for a report from an external firm. In cases where GiveWell is relying on the accuracy of this data for our decision-making (points 2 and 3), I think it makes sense to double-check data accuracy through external data quality audits and/or interviews with local stakeholders.
The same goes with standardizing. We think it's good to have a minimum standard, but we guess there's also variation in what's feasible or makes sense across grantees and expect to figure that out with them. We’ve standardized monitoring and reporting within a few program areas, such as across all our recent water chlorination grants, in collaboration with our grantees (e.g., agreeing on consistent standards on what grantees are expected to track, at what frequency, and how findings should be acted on). However, we haven’t developed monitoring standards that apply across all program areas. I think there could be value in having broader principles, but, so far, we’ve prioritized addressing concrete issues we’ve identified rather than developing these broader principles. We are planning to publish more on the details of these program area-specific monitoring and reporting standards in the near-future.
Hi Tony—This is Mark Walsh, a GiveWell researcher on the team responsible for pressure-testing GiveWell's research processes and conclusions. Thank you for this thoughtful write-up!
At a high level, I agree with the central thesis: we've underinvested in monitoring and evaluation, relative to other components of our analysis. Like you mentioned in your post, we've been working to fix that, and I wanted to share a little bit more on what we've done so far, what we're planning, and some other related gaps in our work.
We think this kind of external engagement with our research is valuable and makes our work better. We'd welcome feedback on the steps we've taken so far and where we should consider doing more.
What we’ve done
Over the past year, we've been working on what we call "M&E red teaming"—a systematic review of the monitoring and evaluation practices of our largest grantees, motivated by the same concerns Tony raises. We're planning to publish our findings soon but wanted to give a brief overview.
From June to December 2025, we dedicated teams of 3-4 research staff to work full-time for roughly 3 weeks each on six program areas: our four top charities plus water chlorination and malnutrition treatment. For each program, we evaluated many of the dimensions Tony highlights in his monitoring checklist: the independence of data collectors from program staff, the neutrality of the sampling frame, the objectivity and precision of measurement approach, data quality checks and backchecks, timeliness of data, triangulation with independent sources, and whether the program is taking timely action to address any issues raised by monitoring.
Since completing the red teaming, we've been doing the following to make improvements:
As Tony suggests, we think the right way to evaluate these investments is based on the value of the information we’ll gain and the impact we think it will have on our future grantmaking decisions. That means that we are more likely to fund expensive M&E in large grantmaking areas for GiveWell (or areas with a lot of room for more funding) and where we have more uncertainty so expect the M&E to affect our grantmaking a lot. For example, we’ve funded expensive independent coverage surveys of insecticide-treated nets and vaccinations programs because we direct a lot of funding to these programs. On the other hand, we are trying to take a rigorous but lighter-touch approach in cases where there is less funding at-stake, or where key uncertainties (e.g., whether an organization can establish partnerships or hire effectively) can be resolved more cheaply before investing in expensive M&E.
What’s next
Over the next several months, we're planning to finalize and roll out our coverage survey standards with grantees; analyze the results from the independent surveys, enhanced M&E, and population/costs work we've commissioned; and dig further on more of these areas (e.g., investigating better ways to monitor the impact of our grantees on disease morbidity and mortality).
I expect we'll learn a lot as we go about what's feasible and what information is actually most valuable. I'm sure our approach will change as we learn more about both the cost and potential impact of this information. That being said, I think we are on the right track.
A broader gap
I also want to flag that we think the issues Tony raises are part of a broader gap. It's not just that we need better quantitative monitoring—we also need to invest more in understanding what's actually happening on the ground with the programs we fund.
We've been trying to gather more "local insights" on our work. This involves site visits, qualitative research, conversations with local experts, and other ways of testing our desk-based assumptions against what’s happening on the ground. One example is funding the Busara Center for Behavioral Economics to observe vitamin A supplementation delivery in Nigeria and interview households, front-line staff, and government officials about the program.
We're still figuring out which approaches are most useful but think shifting more of our research effort "beyond the spreadsheet" in the ways Tony is describing is directionally right and something we’re making progress on. As I said at the start, we welcome feedback on our work so far—and on our future progress as it occurs.
Hi there - thanks for your interest in GiveWell! We don't currently offer informational calls, but we'd encourage you to check out our open roles and apply to any that seem like a good fit for your background and interests.
Hi Nick,
Thanks for noting that section of the post could have been clearer! We’ve edited the article to clarify that New Incentives went from serving 70,000 to 1.5 million children per year.
We agree that the extra lives saved (“indirect deaths” in our analysis) is an interesting question. Both the magnitude of the adjustment and the exact mechanisms (i.e., which other causes those deaths are coming from in the GBD bucket) are major sources of uncertainty in our model, and we don’t currently specify what other deaths are being averted through vaccination in our analysis. We may follow up with a post to share more about our work on indirect deaths in the future.
Thanks again for the feedback!
GiveWell Updates
- GiveWell co-founder and CEO Elie Hassenfeld was recently named to the 2026 TIME100 Philanthropy list. Last year, GiveWell directed over $400 million to programs that save and improve lives, guided by the central question that has driven us from the very beginning—where can a dollar do the most good?
Read More- In a recent podcast episode, GiveWell co-founder and CEO Elie Hassenfeld speaks with Senior Program Officer Adam Salisbury about our growing livelihoods portfolio, focused on increasing the economic well-being of people in extreme poverty. Elie and Adam discuss testing variations of cash transfers, exploring poverty graduation programs, and funding research to answer key questions about which programs are most cost-effective.
Listen Here: Expanding Our Search for Cost-Effective Ways to Reduce Poverty- We recently granted $5 million to the DIV Fund to identify and support promising water quality and access innovations we could consider for future funding. If you're piloting an early-stage water intervention, or know someone who is, the RFP is now open on a rolling basis.
Learn More + Apply- GiveWell has directed more than $1 billion to malaria prevention over our history. With increased capacity on our malaria research team—our largest research subteam—we're expanding our malaria work by funding new research, increasing coverage of core programs, and exploring interventions beyond nets and seasonal malaria chemoprevention.
Read More: Growing GiveWell’s Largest Research Area- Over the last decade, GiveWell has directed nearly $50 million to iron fortification and supplementation programs. In a recent episode of GiveWell Conversations, GiveWell co-founder and CEO Elie Hassenfeld speaks with Researcher Andrew Martin about our work on iron—including how an evaluation of our 2021 grant to Fortify Health, which showed the program was roughly twice as cost-effective as projected, led us to renew our support with a $10 million grant.
Listen Here: What a Decade of Iron Funding Has Taught Us