Hi all,

A friend linked me to the 'red team' contest, and since I undertake cost-effectiveness modelling professionally I thought that would be a useful place I could contribute, potentially.

I'm not an active member of the EA community, so I'd like to ensure I don't straw man the state of the art here; would it be fair to say GiveWell's evaluation of some of its most cost-effective charities represents what the EA community would consider to be a high-quality cost-effectiveness model? If not, what would the EA community consider to be a high-quality cost-effectiveness model in an EA context?

https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/cost-effectiveness-models

(To be clear, the model is great but I think there are a number of areas where it could be improved upon!)

Thanks so much,

Froolow

New Answer
Ask Related Question
New Comment

3 Answers sorted by

I would say that GiveWell's cost-effectiveness analyses are considered excellent (here is a guide from 2019), but they should be taken in context.
From https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/cost-effectiveness-models "we consider our cost-effectiveness numbers to be extremely rough."
"There are many limitations to cost-effectiveness estimates, and we do not assess charities only—or primarily—based on their estimated cost-effectiveness."

And this old blog post: https://blog.givewell.org/2011/08/18/why-we-cant-take-expected-value-estimates-literally-even-when-theyre-unbiased/

They definitely have a much smaller weight than what I first assumed: I initially thought all GiveWell did was to make cost-effectiveness analyses of dozens of charities and recommend the most cost-effective ones. It seems that's completely wrong, and they rely a lot on other criteria and less quantitative or public information.

You might also be interested in the series: Concerns about AMF from GiveWell reading (especially Part 3 and Part 2)

And this FAQ from 2017 from GiveWell

Thank you - really helpful additional information and very useful to have it confirmed that GiveWell are considered high quality models by the EA community. Really appreciate it.

See also

https://forum.effectivealtruism.org/posts/kyJtzRHd6hLzfxshd/announcing-the-legal-priorities-project-writing-competition

I'd say Michael Dickens and Sam Nolan are probably peak performance. 

This is a classic, devastating methodological piece too.

I would guess that other orgs besides GiveWell also have cost-effectiveness models/analyses.

4Gavin2mo
Couldn't find any public OP analyses on a cursory look
4Stefan_Schubert2mo
I guess that if one wants to red team effective altruist cost-effectiveness analyses that inform, e.g. giving decisions, non-public analyses may be relevant.

Thank you - these are really helpful to help me understand. On the Sam Nolan piece especially quantifying uncertainty was one of the biggest critiques I had of the GiveWell model so I'm glad this has already been considered!

2NunoSempere2mo
This has been considered by Sam, but it's still a valid criticism of GiveWell's models as they are now.