When recommending charities, Giving What We Can (GWWC) mirrors the recommendations of their trusted evaluators. For example, they list the Against Malaria Foundation because GiveWell considers it a top charity and the Humane League because Animal Charity Evaluators does. This approach makes sense: consolidating the reviews of multiple evaluators for the GWWC membership and the broader EA community.

One place where this gets tricky is when their evaluators work in the same area but have different funding bars. This is most apparent in global health and development, where GiveWell and Founders Pledge overlap. GiveWell is willing to recommend funding opportunities if they are 5x-8x [EDIT: 10x] better than cash transfers to very poor people while Founders Pledge uses a bar of at least as good as cash transfers.

Reading GWWC's website it wasn't clear to me what they did. For example they badge the salt-iodization charity IGN as "top rated" but not the cash transfer charity GiveDirectly or the deworming charity SCI, even though Founders Pledge recommends all three.

If you look at the Founders Pledge recommendations for GiveDirectly and SCI they start with a note that "This is a Founders Pledge summary and interpretation of original research published by GiveWell." So I initially thought that what was happening was that they would mark the charity as "top rated" if Founders Pledge recommends it based on their own evaluation, but not if Founders Pledge is just relying on GiveWell's evaluation and GiveWell doesn't recommend them.

This seemed bad: even if GiveWell, Founders Pledge, and GWWC all agreed a charity was ~4x more effective than cash transfers, would GWWC rate it below one on which GiveWell had no opinion and Founders Pledge thought was only slightly more cost-effective than cash transfers?

I wrote to GWWC, and there's actually a good system behind these badges. They don't combine the ratings naively, but have their own intermediate threshold at ~3x better than cash transfers. They're able to do this because while Founders Pledge doesn't publish efficacy estimates for charities they do have them internally, and they coordinate with GWWC. This explains why they don't mark GiveDirectly as "top rated": it's part of the baseline by which the other charities are measured. And it also explains why IGN does get the badge: Founders Pledge thinks it's at least 3x better than cash transfers.

What it doesn't explain, though, is why SCI isn't listed. In November 2021 GiveWell estimated (sheet, via) that donations to SCI were about 13x as effective as cash transfers. The reason GiveWell stopped recommending SCI as a top charity was not that this estimate changed, but that GiveWell changed their criteria:

We have always supported deworming because we estimate it to be highly cost-effective—deworming programs are very inexpensive, and we think there's a small chance they may lead to large income gains later in life. But we're more uncertain about deworming than we are about our current top charities, so deworming doesn't fulfill the second criterion on our list—a high likelihood of substantial impact. However, we expect to continue supporting deworming through grants from our All Grants Fund, as there continue to be funding opportunities in deworming that exceed our cost-effectiveness threshold, and we encourage donors who have supported individual deworming programs in the past to keep doing so

In their response to my questions GWWC pointed me to their note on their page for SCI where they said "We intend to reevaluate the status of all of the charities listed on our platform that are working on deworming programmes after one of our trusted evaluators, Founders Pledge, publishes its review of deworming's cost-effectiveness." They also told me that until Founders Pledge finishes their re-evaluation of deworming they don't want to list them as "top rated". Unless they have some reason to think that GiveWell's estimate of SCI's cost effectiveness has fallen by a factor of four in the last year, however, I don't think the decision to withhold this designation makes sense. GiveWell dropping them as a top charity was a decision about how to handle risk and uncertainty, and GWWC accepts Founders Pledge recommendations which both (a) include some reviews much older than 2021 and (b) are more accepting of risk than GiveWell was even before the August 2022 changes.

Aside from the questions about how SCI and other deworming charities should be badged, however, my main takeaway here is that I'd like to see more public details from GWWC about this process in their inclusion criteria. Specifically, I'd like to see them describe their ~3x cash threshold and use of Founders Pledge internal estimates. I'd also like to see Founders Pledge make their estimates public, or at least publish which of their recommendations meet the 3x bar. The idea driving this is that, as I wrote earlier this week, I think the "recommending specific charities" portion of GWWC's work should depend only on public information.


Disclosure: my wife used to be President of GWWC and is on the GiveWell board, but I haven't run this post by her and I don't know what she thinks here. I sent a draft of this post to GWWC, Founders Pledge, and GiveWell; thanks to Sjir at GWWC for his substantial feedback, and Matt at Founders Pledge for his quick responses.

83

Comments4
Sorted by Click to highlight new comments since: Today at 6:06 PM

Hi Jeff, thanks for another helpful suggestion! (previous one) In this case, I agree there is room for improvement, and we'll aim to update our inclusion criteria throughout 2023 and to provide more details where we can. The reasons this particular thing isn't in there yet are (1) we simply haven't prioritised writing this out yet, as it is quite detailed/applies to just one of the four "cause areas" we cover and to just one evaluator (FP) and it would require quite a bit of extra context to explain to the broad audience we try to reach (e.g. laying out what we mean by these ratings, what is measured, limitations etc.) which was beyond the scope we were able to set for this giving season (during which we already had a lot of things to update with a small team) (2) this relies on FP's internal ratings and I'm not sure whether FP would want these to be public, e.g. given how rough they are/how much context they need/to avoid over-updating, but I'll leave it to them to respond on that point.

On SCI specifically, my recollection is that GiveWell's November 2021 analysis is no longer current/that SCI has made significant changes to its programme since that evaluation was done, though I'm not 100% sure. In any case, for the deworming charities more generally we decided to stick closely with our inclusion criteria, which meant not recommending them as top-rated (because they don't clear GW's nor FP's current bar at this moment/they weren't recommended to us by either) and listing until we receive FP's updated estimates. This seemed the better option in particular because we know FP will have updated estimates relatively soon, and IIRC they don't expect all deworming charities to necessarily clear the 3x bar (though again referring to them here for a response, if they are willing to comment before finalizing the evaluation). Hope that clarifies!

GiveWell's November 2021 analysis is no longer current/that SCI has made significant changes to its programme since that evaluation was done

Since you wrote this, GiveWell's January 2023 update is out; they have SCI at ~13.5x cash.

Thanks for pointing at this! We'll make sure to ask GW about this at our next point of contact -i.e. whether they think we should recommend SCI/deworming charities given our different bar and their cost-effectiveness analysis - and this may lead us to change the status of these charities.

Thanks for clarifying!

If FP does not recommend SCI right now it would be nice if they noted that; right now they just have "Please note this page was last updated in 2018. While our overall views remain unchanged, some details may be out of date." But that's an issue for FP, not you!