I'm just a normal, functioning member of the human race, and there's no way anyone can prove otherwise.
Principal Analyst at SoGive
I'm not sure I follow your point about volumes. The cost-effectiveness model is for those who receive the net. There's no need to dilute the impact on these people merely because other people don't experience the same impact. You just say 'this is the benefit to these people, achieved at this cost'.
Thanks for your response - and more generally, thanks for putting time and effort into scrutinising GiveWell's analysis, and sharing your views here.
Net distributions cover the whole community, they are not targetted at just under-5s. Using GiveWell's figures, 16% of the population is under 5. Scaled up 1.8 people per net this suggests coverage for 1.4m * 16% * 1.8 = 0.4m young children.
You're of course right. I originally wrote 'people' rather than 'children', but changed it because the discussion was focused on under 5 mortality.
That's not going to materially change the mortality rate in a country with c.12m under 5s.
Sure - but the question is whether it changes the mortality rate of those receiving the bednets.
I can confirm that the 7.7 mortality rate is an unadjusted country-wide mortality rate and 11.9 is the rate GiveWell estimates would occur with no distributions from AMF.
I think you may be right, and it seems like GiveWell may have made a mistake here. But that doesn't mean the mortality rate would be unchanged for those who receive (or would receive) bed nets. Rather, as I suggested before:
When considering the impact of a donation to AMF, we should compare the expected mortality benefit if AMF distributes bednets compared to if they do not. According to their website, AMF did not make any significant bednet distributions before 2019, with just 1.4m nets across 2014-2016 for a population of around 75m. This means the counterfactual for AMF not making distributions in future is the same as the past, and that the current mortality rate of 7.7 per 1,000 child years is maintained. There is no reason to consider an increase to 11.9 or any other number if there are no future AMF distributions since there have been almost no past AMF distributions in this country
If AMF distributed 1.4m nets across 2014-16, then that's a lot of children with nets. Say 2.8m, if it's 2 children per net. If nets work, then these children will be protected to some extent, and have reduced mortality from malaria. An absence of future AMF bednet distributions (and an absence of an alternative) would result in increased mortality for these children.
Now, there's the question of whether Givewell are right to indicate mortality would increase from 7.7 to 11.9. If these are country-level figures, in a country which mostly doesn't have bednets, then plausibly mortality for those who do have bednets is actually lower than the country-level average of 7.7. Then, if the AMF bednets are stopped, we might expect an increase in mortality back up to the country average of 7.7. However, it may be that Givewell have already adjusted for this (I haven't looked into it), and actually the 11.9 is indeed the country-level figure that the mortality rate would be expected to increase back up to.
(a minor point - it would be helpful if you edited this to indicate you're discussing the Democratic Republic of Congo; I initially thought you were making claims about AMF's total distributions)
"The one you should use depends on context. It should depend on how much you care about false positives vs false negatives in that particular case"
Yep, exactly! Assume you're a doctor, have a bunch of patients with a disease that is definitely going to kill them tomorrow, and there is a new, very low-cost, possible cure. Even if there's only one study of this possible cure showing a p-value of 0.2, you really should still recommend it!
To some extent SoGive will be implementing what you're suggesting. As well as the overall top, EA-recommended charities, we are also looking to identify the best charities within other cause areas (e.g. poverty/homelessness in the UK, developed world health, tree-planting charities). Ideally we want to nudge people to switch to the overall top charities regardless of cause area, but we know that a lot of people are very committed to a particular cause, so it could be quite valuable to help them at least identify the top charities within that cause.
GiveWell and Founder's Pledge both do excellent work, so I don't think it would be right to suggest SoGive's approach is fundamentally better - indeed we often build on the work of these two organisations. However, as you say, there is some value in having multiple independent perspectives on a topic.
We are aiming to fill a neglected niche, namely the application of an EA/cost-effectiveness approach to a much broader set of charities than those of most other EA organisations. Think Charity Navigator, but with a focus on impact rather than mostly-irrelevant financial metrics. We think there is scope to nudge a large number of people (most who otherwise won't be aware of EA) to support higher impact charities within and across cause areas, by including a comparison with many of the well-known charities in the UK.
Relatedly, there are also particular topics/cause areas where there is a lot of public interest, but that existing EA orgs have concluded probably aren't going to include the very best charities.
As an example, we are currently undertaking a review of tree-planting charities. It seems unlikely that the best tree-planting charity will be as cost-effective as (e.g.) the Clean Air Task Force when it comes to averting/reducing CO2eq. But there is a lot of interest in tree-planting, both from individuals and corporations. We hope that by having tree-planting charities alongside the likes of CATF, at least some people who are interested in tree-planting will switch donations to CATF (because they actually care about CO2eq), whereas others who (for whatever reason) really really only want to plant trees, will at least switch to the best tree-planting charity.
I'm definitely in favour of further consideration of this. However, I'd like to see the case for curing infectious diseases considered alongside the case for researching anti-ageing interventions.
It seems plausible that developing a successful anti-ageing intervention (a) would have an impact larger in scale than one for infectious disease (because it would be cross-cutting against the risk of cancer, heart disease, stroke, dementia, worsening mobility etc) (b) is more neglected (unlike research into treatments for specific diseases of ageing) (c) would also reduce deaths from some infectious diseases (e.g. influenza, Covid) (d) is much more risky/uncertain in terms of tractability
On LinkedIn Ben Todd repeated his claim about room for more funding up until the end of 2023, based on this GiveWell spreadsheet
I'll repeat my reply here:
I'd really like to hear from someone at GiveWell (or the specific charities) to verify that this is the right interpretation of the funding gaps. For example, presumably this considers the funding gaps for specific programmes/countries that the likes of AMF are currently focusing on. But once that funding gap is filled, it seems plausible that there are other countries they could work on.As an example, AMF currently state they have a funding gap of $53m (vs less than $1m according to that GiveWell spreadsheet). They state that "Agreements are being finalised with each country's Ministry of Health. This process is far advanced for the above programmes and we do not anticipate any issues. We do not publicly identify countries involved until an Agreement is signed". It seems plausible that these countries are not included in GiveWell's figures.
Another possibility is that the funding gap on AMF's website is for programmes that will be implemented after 2023
Hi Saulius, I've done 3 very basic estimates here:
To get e.g. more than 20% probability, it seems like you'd have to make some very bad assumptions (weirdly high base rates of Covid amongst presumptive attendees, combined with incompetence or malice when it comes to testing). Seems more like 1-5% risk.
Have you read this GiveWell page on bed nets? They state:
There is strong evidence that when large numbers of people use LLINs to protect themselves while sleeping, the burden of malaria can be reduced, resulting in a reduction in child mortality among other benefits.
Or this Cochrane review?
Insecticide‐treated nets reduce child mortality from all causes by 17% compared to no nets (rate ratio 0.83, 95% CI 0.77 to 0.89; 5 trials, 200,833 participants, high‐certainty evidence). This corresponds to a saving of 5.6 lives (95% CI 3.6 to 7.6) each year for every 1000 children protected with ITNs. Insecticide‐treated nets also reduce the incidence of uncomplicated episodes of Plasmodium falciparum malaria by almost a half (rate ratio 0.55, 95% CI 0.48 to 0.64; 5 trials, 35,551 participants, high‐certainty evidence) and probably reduce the incidence of uncomplicated episodes of Plasmodium vivax malaria (risk ratio (RR) 0.61, 95% CI 0.48 to 0.77; 2 trials, 10,967 participants, moderate‐certainty evidence).
If the nation-level data isn't supportive of this, then perhaps this is worthy of further investigation to understand why it may be different from the trials.
You seem to acknowledge this by saying 'Maybe the RCT evidence is so convincing that the noise of country-level data doesn’t matter' - but if your claim is that there is 'no evidence of impact' specifically at the country-level, then I'd encourage you to be clear about this with your heading. The statement that 'when you try to measure outputs there is no evidence of impact' doesn't seem true.