Hi Froolow,

Thank you so much for this thoughtful review!

We've looked through the issues you identify as errors, and we believe that each of those is either not an error or is something we've deprioritized because addressing it would not change our grantmaking decisions. You can see our analysis here.

This only applies to the potential errors you pointed out, not your higher-level feedback or judgment calls. We're grateful for your engagement here, and we are working on improving the architecture of our CEAs, in line with your higher-level feedback. We also recognize that the interpretability of our cost-effectiveness analysis is an area we need to improve, and we're working on that as well.

Since the linked document was written for internal use, it hasn't been fully vetted in the same way that what we publish on our website is. However, we thought it may be useful to share in its current form.

Best,

Andrew (GiveWell)

I’m glad you found this response helpful!

We expect to wait until the results of FEM’s RCT are available before deciding whether to recommend funding for the program itself; we do have some remaining uncertainties, and it's possible that additional work could lead to significant changes in our cost-effectiveness estimate. We also plan to continue our own research on family planning programs in general, which may affect our funding decisions in this area.

As our investigations progress, we’ll share updates on our website. Please feel free to reach out directly if you have questions in the meantime!

Best,

Roman Guglielmo

Donor Relations Associate

GiveWell

Hi Michael,

Thank you for raising these points! We’ve shared your thoughts with our research team for future consideration.

If you have any other questions or concerns about our work, please feel free to reach out to us!

Roman Guglielmo

Donor Relations Associate

GiveWell

Hi!

Thanks for this comment. We are interested in potentially funding family planning programs, including Family Empowerment Media. Our research and grant decisions are independent of our outreach and fundraising plans. To clarify and add detail on some of the points above:

GiveWell recommended a $500,000 grant to Family Empowerment Media in March 2023. This funding would support a planned randomized controlled trial (RCT) of its program. We haven't yet published about this grant, but we plan to soon. (Our grant page publication often lags our funding decisions considerably, though this is something we're working to improve!)

We believe that FEM or other types of family planning–related programs may be very cost-effective—our current rough estimate for FEM's program (not the RCT of the program) puts it at 18-28 times as cost-effective as unconditional cash transfers—but we have a high degree of uncertainty about some of our inputs and expect that the results of FEM's RCT would help us address some of our uncertainties. We are also continuing to assess how to prioritize family planning programs among the many other promising programs we're investigating.

I hope that's helpful!

Best,
Miranda Kaplan
GiveWell Communications Associate

Hi, Hakon,

We were very excited to learn of this trial, a much-needed step toward reducing deaths from tuberculosis. We applaud Gates and Wellcome for this sizable commitment of resources, which is, as Nick points out, necessary to gain adequate information about the vaccine's efficacy, but beyond the reach of most funders.

We have investigated TB-related funding opportunities, and we remain very open to funding either programs or research. However, we have several significant uncertainties about the programs we've explored so far and have not yet funded any at scale.

We are working on a report summarizing the evidence for mass TB screen-and-treat programs, which will provide more detail on our views specific to that type of program.

Best, Miranda

Hi, Nick,

Yes, to clarify, MiracleFeet is selecting the on-the-ground evaluator that will conduct the monitoring in each location, although GiveWell will lightly vet MiracleFeet's choices and meet with each evaluator.

The primary purpose of the monitoring grant is to understand how many children are treated for clubfoot both with and without MiracleFeet's support. So, although MiracleFeet has records of children treated through facilities it's supported, we also want an assessment of baseline treatment coverage before MiracleFeet launches its program (or expands it, in the case of the Philippines). We do plan to incorporate some form of data audit as part of endline activities; we'll work out the details of that at a later date.

Thanks again for your interest in this and for taking the time to ask questions!

Miranda

Hi, Nick,

Thanks for your comments, your insight into this grant, and your support! 

We do expect to get input from local hospital staff on existing treatment coverage through the baseline surveys. The monitoring grant will fund the creation of a sampling frame that includes both public and private health facilities, which we think will yield more complete data than contacting hospitals through our partners.

We agree that potential bias from external evaluators is a risk for the reasons you've mentioned. While we won't be involved in the selection of evaluators, we plan to do the following to mitigate that risk:

• Meet with representatives of all the firms and vet them at a high level so we can identify red (and green) flags. 
• Work closely with both MiracleFeet and the external evaluators throughout the process, so that we can ask questions about and provide input on their research strategies along the way.

We don't think this will completely eliminate uncertainty about the quality of monitoring results, but we expect it will help. We also think there is some value to be gained from working with evaluators who have a strong familiarity with the local context.

I hope that's helpful!

Best,
Miranda

Hi, J.T.,

Sorry for the delay in getting back to you! 

In an ideal world, we would have included financial burden for tobacco in these rough calculations (which were mostly intended to narrow down what we'd focus on in our initial foray into public health regulation grants). But essentially, the last line of your response is the answer: because we value children's lives so highly, we expected that incorporating the financial burden of tobacco use would not be enough for us to prioritize it over lead. Lead specifically affects young children, and the gap in estimated spend per unit of burden ($0.07 for lead versus $0.24 for tobacco) is large. If we were doing a deeper investigation of tobacco policy as a possible funding area, we would aim to more thoroughly account for its harms. 

Best,

Miranda

Hi, J.T.,

As shown by the research conversation notes you link to, GiveWell lightly explored funding tobacco control policy advocacy in the past. Ultimately we decided to focus on other policy areas (alcohol policy, lead exposure, and self-harm from pesticide ingestion) that appeared more neglected. You can read more about our public health regulation research on this page—see "Cause areas we investigated at a shallow level and deprioritized," and this spreadsheet, linked from footnote 24, which gives our estimates of how much is spent on tobacco policy (and other causes we looked into) per unit of burden.  

Thank you for the thought-provoking post!

Best,

Miranda Kaplan
GiveWell Communications Associate