Hide table of contents

Update 2021-10-06: I believe I was overconfident in my original interpretations of these Metaculus questions. Some commenters pointed out alternative interpretations of people's answers that could allow us to draw orthogonal or opposite conclusions. For example, on question 1, Metaculus users might predict GiveWell's top charities to drop off the list by 2031 not because better charities are discovered, but because current charities run out of room for more funding.

In the giving now vs. later debate, a conventional argument in favor of giving now is that people become better off over time, so money spent later will do less good. But some have argued the opposite: as time passes, we learn more about how to do good, and therefore we should give later. (Or, alternatively, we should use our money now to try to accelerate the learning rate.)

Metaculus provides some evidence that the second argument is the correct one: money spent later will do more good than money spent now.

This evidence comes from two Metaculus questions:

  1. Will one of GiveWell's 2019 top charities be estimated as the most cost-effective charity in 2031?
  2. How much will GiveWell guess it will cost to get an outcome as good as saving a life, at the end of 2031?

A brief explanation for each of these and why they matter:

On question 1: As of July 2021, Metaculus gives a 30% probability that one of GiveWell's 2019 top charities will be ranked as the most cost-effective charity in 2031. That means a 70% chance that the 2031 charity will not* be one of the 2019 recommendations. This could happen for two reasons: either the 2019 recommended charities run out of room for more funding, or GiveWell finds a charity that's better than any of the 2019 recommendations. This at least weakly suggests that Metaculus users expect GiveWell to improve its recommendations over time.

On question 2: Metaculus estimates that GiveWell's top charity in 2031 will need to spend $430 per life saved equivalent (according to GiveWell's own analysis). For comparison, in 2019, GiveWell estimated that its most cost-effective charity spends $592 per life saved equivalent. (These figures are adjusted for inflation.)

As with question 1, this does not unambiguously show that GiveWell top charities are expected to improve over time. Perhaps instead Metaculus expects GiveWell's estimate is currently too pessimistic, and it will converge on the true answer by 2031. But the cost reduction could also happen because GiveWell top charities truly get more effective over time.

Some caveats:

  1. These Metaculus answers only represent the opinions of forecasters, not any formal analysis. (Some forecasters may have incorporated formal analyses into their predictions.)
  2. Neither question directly asks whether money spent in 2031 will do more good than money spent now. (I don't know how to operationalize a direct question like that. Please tell me if you have any ideas.)
  3. These questions only ask about GiveWell top charities. Even if GiveWell recommendations become more effective over time, the same might not be true for other cause areas.

20

0
0

Reactions

0
0
New Answer
New Comment

6 Answers sorted by

It's worth pointing out that these questions apply specifically to global health and development, but could be very different in other cause areas.

I don't think question 1 provides evidence that money will do more good in the future. It might even suggest the opposite: As you point out, malaria prevention and deworming might run out of room for more funding, and to me this seems more likely than the discovery of a more cost-effective option that is also highly scalable (beyond >$30 million per year).

Relevant: https://www.metaculus.com/questions/1332/will-global-malaria-mortality-rates-be-reduced-by-90-when-compared-with-2015-rates-by-2030/

Metaculus users give a 52% chance that global malaria deaths will be 90% lower in 2030 than they were in 2015. If that is the case, then I would expect their now highly rated malaria charities to be less cost effective than they are now.

Cool questions! I am a bit hesitant to update much:

  • they don’t seem to be too active, e.g. few comments, interest count at around 10 (can you see the number of unique forecasters somehow?)
  • the people doing the forecasts are probably EA adjacent and if they did something akin to a formal analysis, they would share it with the EA community, or at least in the comments, as it seems relatively useful to contribute this

For the first question, you can see under "community stats" the number of unique users, currently 28. For the second one you cannot see it on the page, I'm not sure why, but I'd guess its a similar ratio (I.e. approx half of the number of predictions)

2
MaxRa
3y
Thanks!

Is there some kind of up-to-date dashboard or central source for GiveWell's main "cost-per-expected-life" figure? 

  • The Metaculus question mentioned in this post cites values like $890 in 2016,  $823 in 2017, $617 in 2018 and $592 in 2019, and I can't find the field they refer to in the resolve condition (?!)
  • This 80K article lists the value as $2300 in 2020.
  • This GiveWell summary sheet from 2016 has a minimum value of $901
  • GiveWell's Top Charities page lists $3000-$5000 to save a life for Malaria Consortium, Against Malaria Foundation, New Incentives, and Hellen Keller International.

If such a thing does not exist, I'll probably reach out to GiveWell and see what they think about implementing one. There are so many numbers floating around that are hard to verify and differ dramatically.

(1) The Metaculus question adjusts numbers for inflation to 2015 dollars, so they wouldn't appear explicitly in GiveWell's spreadsheets.

(2) Note that there's a distinction between "outcome as good as saving a life" and "cost per life saved". The $890 number is (GiveWell's 2016 estimate of) the former while the $3,000 - $5,000 is the latter. The former includes good done by reducing the probability that people die as well as good done by raising peoples' incomes, which at some point is equivalently good to averting a death.

Is there some kind of up-to-date

... (read more)

I mistakenly submitted this as a question instead of as a post. Is there any way to convert it to a post?

Givewell charities are expected to increase local GDP, and have other effects which are expected to improve wellbeing over the long run. Starting these 2nd order effects sooner can offset the additional cost of doing them now (if that exists).

Comments8
Sorted by Click to highlight new comments since: Today at 6:08 PM

For the first question, I was one of the forecasters who gave close to the current Metaculus median answer (~30%). I can't remember my exact reasoning, but roughly:

1. Outside view on how frequently things have changed + some estimates on how likely things are to change in the future, from an entirely curve fitting perspective.

2. Decent probability that the current top charities will go down in effectiveness as the problems become less neglected/we've had stronger partial solutions for them/we discover new evidence about them. Concretely:

Malaria: CRISPR or vaccines.  But also I place decent probability on bednet production and distribution being fully solved by states/international actors/large NGOs.
Deworming:  Possibilities include a) we uncover new evidence that suggests deworming is less effective than we previously thought, especially at 2021/2030 worm loads* or b) mass deworming decreases the total amount of worms, decreasing marginal value, or c) both.
Cash: At GiveWell scale, I don't think direct cash transfers is ever competitive with your current best guess for top health/development interventions under naive cost-effectiveness unless you apply a strong robustness penalty to everything else. 
Vitamin A: haven't thought much about it tbqh, so retreat to priors.
 

*The evidentiary base for deworming was always shaky to begin with, I think (within the randomista paradigm) it's reasonable to model deworming as a relatively high-risk high-reward economic intervention, and we may eventually discover evidence that conclusively demonstrates that deworming isn't economically that effective.

This could happen for two reasons: either the 2019 recommended charities run out of room for more funding, or GiveWell finds a charity that's better than any of the 2019 recommendations.

I think the third reason that you mention later is more likely, namely that "Perhaps instead Metaculus expects GiveWell's estimate is currently too pessimistic, and it will converge on the true answer by 2031."

This seems most likely to me since my guess is that the reason why GiveWell's estimate changed from $890 in 2016 to $436 in 2019 is because their methodology or something else changed (e.g. 2019-GiveWell thought 2016-GiveWell's way of estimating the "cost to get an outcome as good as saving a life" over-estimated the true cost). I don't think they actually believe the best giving opportunity was twice as cost-effective in 2019 as 2016. (I don't actually know this for sure, so if you or someone else knows better, please correct my guess.)

This at least weakly suggests that Metaculus users expect GiveWell to improve its recommendations over time.

Does it? If "the 2019 recommended charities run out of room for more funding" this could be because e.g. governments or billionaires have stepped in to make sure that every child who needs one has a bed net and all deworming efforts have been fully funded forever. It seems to me that this would mean that the marginal philanthropist no longer has access to this low-hanging fruit. Since these opportunities would be fully-funded, GiveWell would then have to recommend the next best thing, which would be less cost-effective in expectation, not more.

While my comments reveal that I disagree with the inferences you make in this post, I just want to say that I think this sort of analysis is really cool and I appreciate your thoughtfulness with coming up with these questions and writing this post. Your writings have influenced my thinking and changed my view on several occasions and I look forward to reading more of what you have to say in the future.

For comparison, in 2019, GiveWell estimated that its most cost-effective charity spends $592 per life saved equivalent. (These figures are adjusted for inflation.)

I think you said in 2020 the $592 number was a mistake:

Correction: I copy/pasted the cost per life saved numbers from the previous question, but the $592 number for 2019 is as of the middle of the year. The end-of-year estimate (adjusted to 2015 dollars) is $436.

$436 is a lot closer to $430 than $592.

Meta question: Is there a reason you made this post a Question post rather than a regular post? Seems like it should be a regular post.

It was an accident. I should have made a post, not a question.

I bought the Metaculus predictions (using my Tachyon points on Metaculus) for each of the two questions in your post as well as the malaria question Charles Dillon linked to.

For those unfamiliar, the Metaculus prediction has a better track record than the community prediction (public to everyone, which Michael cited in his post as what Metaculus thinks) over questions on the Metaculus site, however is not generally visible until forecasts on a question are closed.

Knowing the Metaculus predictions does not seem very insightful to me. Nevertheless, if anyone would care to know them, feel free to DM me.

Metaculus estimates that GiveWell's top charity in 2031 will need to spend $430 per life saved equivalent (according to GiveWell's own analysis).

Assuming I'm not mistaken about how Metaculus works, $430 is the median of recent forecasters' median forecasts on this question. But don't we really care more about the mean expected cost-effectiveness than the median? (I could be wrong about this since after all (I think) GiveWell's estimate is the median output of their staff members' inputs into their cost-effectiveness model.)

Metaculus doesn't give the mean value, but looking at the shape of the probability density distribution on the Metaculus question, it looks to be roughly symmetrical about the median and given that the axis is logarithmic, I think this means the mean value is greater than the median.

So assuming the mean is what we care about, then it seems that Metaculus community prediction is greater than GiveWell's 2019 estimate, suggesting that money (given to GiveWell charities) will do less good in the future rather than more.

Curated and popular this week
Relevant opportunities