GWWC's 2020–2022 Impact evaluation (executive summary)

Giving What We Can (GWWC) is on a mission to create a world in which giving effectively and significantly is a cultural norm. Our research recommendations and donation platform help people find and donate to effective charities, and our community — in particular, our pledgers — help foster a culture that inspires others to give. 

In this impact evaluation, we examine GWWC's^[1] cost-effectiveness from 2020 to 2022 in terms of how much money is directed to highly effective charities due to our work.

We have several reasons for doing this: 

• To provide potential donors with information about our past cost-effectiveness.
• To hold ourselves accountable and ensure that our activities are providing enough value to others.
• To determine which of our activities are most successful, so we can make more informed strategic decisions about where we should focus our efforts. 
• To provide an example impact evaluation framework which other effective giving organisations can draw from for their own evaluations.

This evaluation reflects two months of work by the GWWC research team, including conducting multiple surveys and analysing the data in our existing database. There are several limitations to our approach — some of which we discuss below. We did not aim for a comprehensive or “academically” correct answer to the question of “What is Giving What We Can’s impact?” Rather, in our analyses we are aiming for usefulness, justifiability, and transparency: we aim to practise what we preach and for this evaluation to meet the same standards of cost-effectiveness as we have for our other activities.

Below, we share our key results, some guidance and caveats on how to interpret them, and our own takeaways from this evaluation. GWWC has historically derived a lot of value from our community’s feedback and input, so we invite readers to share any comments or takeaways they may have on the basis of reviewing this evaluation and its results, either by directly commenting or by reaching out to [email protected].

Key results

Our primary goal was to identify our overall cost-effectiveness as a giving multiplier — the ratio of our net benefits (additional money directed to highly effective charities, accounting for the opportunity costs of GWWC staff) compared to our operating costs. ^[2]

We estimate our giving multiplier for 2020–2022 is 30x, and that we counterfactually generated $62 million of value for highly effective charities.

We were also particularly interested in the average lifetime value that GWWC contributes per pledge, as this can inform our future priorities.

We estimate we counterfactually generate $22,000 of value for highly effective charities per GWWC Pledge, and $2,000 per Trial Pledge.

We used these estimates to help inform our answer to the following question: In 2020–2022, did we generate more value through our pledges or through our non-pledge work? 

We estimate that pledgers donated $26 million in 2020–2022 because of GWWC. We also estimate GWWC will have caused $83 million of value from the new pledges taken in 2020–2022.^[3]

We estimate GWWC caused $19 million in donations to highly effective charities from non-pledge donors in 2020–2022.

These key results are arrived at through dozens of constituent estimates, many of which are independently interesting and inform our takeaways below. We also provide alternative conservative estimates for each of our best-guess estimates.

How to interpret our results

This section provides several high-level caveats to help readers better understand what the results of our impact evaluation do and don’t communicate about our impact. 

We generally looked at average rather than marginal cost-effectiveness

Most of our models are estimating our average cost-effectiveness: in other words, we are dividing all of our benefits by all of our costs. We expect that this will not be directly indicative of our marginal cost-effectiveness — the benefits generated by each extra dollar we spend — and that our marginal cost-effectiveness will be considerably lower for reasons of diminishing returns. 

We try to account for the counterfactual

This evaluation reports on the value generated by GWWC specifically. To do this, we estimate the difference in what has happened given GWWC existed, and compare it to our best guess of what would have happened had we never existed (the “counterfactual” scenario we are considering).

We did not model our indirect impact

For the purpose of this impact evaluation, we focused on Giving What We Can as a giving multiplier. Our models assumed our only value was in increasing the amount of donations going to highly effective charities or funds. While this is core to our theory of impact, it ignores our indirect impact (for example, improving and growing the effective altruism community), which is another important part of that theory.

Our analysis is retrospective

Our cost-effectiveness models are retrospective, whereas our team, strategy, and the world as a whole shift over time. For example, in our plans for 2023 we focus on building infrastructure for long-term growth and supporting the broader effective giving ecosystem. We think such work is less likely to pay off in terms of short-term direct impact, so we expect our giving multiplier to be somewhat lower in 2023 than it was in 2020–2022.

A large part of our analysis is based on self-reported data

To arrive at our estimates, we rely a lot on self-reported data, and the usual caveats for using self-reported data apply. We acknowledge and try to account for the associated risks of biases throughout the report — but we think it is worth keeping this in mind as a general limitation as well.

The way we account for uncertainty has strong limitations

We arrived at our best-guess and conservative multiplier estimates by using all of our individual best-guess and conservative input estimates in our models, respectively. Among other things, this means that our overall conservative estimates underestimate our impact, as they rely on many separate conservative inputs being correct at the same time, which is highly unlikely.

We treated large donors’ donations differently

For various reasons, we decided to treat large donors differently in our analysis, including by fully excluding our top 10 largest GWWC Pledge donors from our model estimating the value of a new GWWC Pledge. We think this causes our impact estimates to err slightly conservatively. 

We made many simplifying assumptions

Our models are sensitive to an array of simplifying assumptions people could disagree with. For instance, for pragmatic reasons we categorised recipient charities into only two groups: charities where we are relatively confident they are “highly effective,” and charities where we aren’t.

We documented our approach, data, and decisions

In line with our aims of transparency and justifiability, we did our best to record all relevant methodology, data, and decisions, and to share what we could in our full report, working sheet, and survey documentation. We invite readers to reach out with any requests for further information, which we will aim to fulfil insofar as we can, taking into account practicality and data privacy considerations.

Our takeaways

Below is a selection of our takeaways from this impact evaluation, including implications that could potentially result in concrete changes to our strategy. Please note that the implications — in most cases — only represent updates on our views in a certain strategic direction, and may not represent our all-things-considered view on the subject in question. As mentioned above, we invite readers who have comments or suggestions for further useful takeaways to reach out.

Our giving multiplier robustly exceeds 9x

• Our conservative estimate for our multiplier — which combines multiple conservative inputs — is 9x, suggesting that every $1 spent on GWWC in 2020–2022 is highly likely to have caused more than $9 to go to highly effective charities — and our best guess is much higher, at $30.
• This doesn’t imply funding GWWC on the margin will always yield such a high multiplier, but it does provide a strong case for scaling up some of GWWC’s most cost-effective current activities — or at least for making sure we are able to keep doing what we do well.

New GWWC Pledges likely account for most of our impact

• We estimate we generated 1.5-4.5x^[4] more value through the GWWC Pledge than through non-pledge donations. 
• This can inform our work going forward — for example, on our marketing strategy and on whether we choose to support other effective giving organisations with pledge-related products.

We found an increase in recorded GWWC Pledge donations with time

• One of our most surprising findings was that the average recorded yearly donations of GWWC Pledge donors seems to increase with Pledge age rather than decrease: the increase in average donations among the Pledge donors who keep recording their giving more than compensates for any drop-out of Pledge donors. We have several hypotheses for why this may be and are not confident it will replicate in future evaluations.
• This is weak evidence in favour of promoting pledges among younger audiences with high earning potential, as retention is less of an issue than one might expect.

A small but significant percentage (~9%) of our Trial Pledgers have gone on to take the GWWC Pledge, and this plausibly^[5] represents the bulk of the value we add through the Trial Pledge

• This is evidence that our Trial Pledge is working as a step towards the GWWC Pledge, but there is also room to improve.
• Given our $22,000 estimate of the value we cause per GWWC Pledge, and the fact that we see a similar number of Trial Pledges and GWWC Pledges each year, this suggests there is value in experimenting with activities that could increase the proportion of Trial Pledgers who take the GWWC Pledge.

The vast majority of our donors give to charities that we expect are highly effective

• We estimate that more than 70% of all pledge and non-pledge donations we recorded went to organisations that would meet the criteria to be one of our top-rated charities and funds at the time.
• This suggests a lot of our value comes from highlighting and facilitating donations to these charities and funds in particular, which is a relevant consideration for — among other things — updating our inclusion criteria.

Our donations follow a heavy-tailed distribution

• Less than 1% of our donors account for 50% of our recorded donations. This amounts to dozens of people, while the next 40% of donations (from both pledge donors and non-pledge donors) is distributed among hundreds. This suggests that most of our impact comes from a small-to-medium-size group of large donors (rather than from a very small group of very large donors, or from a large group of small donors).^[6]
• This information can be relevant in deciding which groups we could put extra resources into reaching to increase our giving multiplier — though it’s worth noting that our short-term giving multiplier isn’t our only consideration for deciding which groups we target.

Nearly 60% of our donors’ recorded donations go to the cause area of improving human wellbeing

• For the remaining cause areas, slightly below 10% of recorded donations go improving animal welfare, slightly above 10% to creating a better future, and another ~20% to unknown or multiple cause areas.
• We don’t currently have a clear view of what (if anything) this finding implies we should do — as we generally don’t prioritise any of these cause areas over one another — but we think it can inform discussions on how much we highlight each area in our communications, and how we prioritise our time when investigating our recommendations.

We plan to report on how we have used these takeaways in our next impact evaluation, both to hold ourselves accountable to using them and to test how useful they turned out to be.

Where you can learn more

• Our full impact evaluation report — this includes a much more detailed description of our assumptions, considerations, methodology, and results.
• Our working sheet — this contains all of our estimates and how we arrived at them.
• Our survey documentation and results — this documents how we conducted the 2023 surveys that informed our evaluation, and summarises our findings.

How you can help

If you are interested in supporting GWWC’s work, we are currently fundraising! We have ambitious plans, and we’re looking to diversify our funding and to extend our runway (which is currently only about one year). For all of this, we are looking to raise ~£2.2 million by June 2023, so we would be very grateful for your support. You can read our draft strategy for 2023, make a direct donation, or reach out to our executive director for more information. 

Acknowledgements

We’d like to thank the many people who provided valuable feedback — including Anne Schulze, Federico Speziali, Basti Schwiecker, and Jona Glade — and in particular Joey Savoie, Callum Calvert, and Josh Kim for their extensive comments on an earlier draft of this evaluation. Thank you also to Vasco Amaral Grilo for providing feedback on our methodology early on in our process, and for conducting his own analysis of GWWC's impact.

We’d also like to give a special thank you to our colleagues Fabio Kuhn and Luke Freeman for their extensive support on navigating our database, to Luke also for support on our survey design, to Katy Moore for high-quality review and copy editing, to Nathan Sherburn for helping us compile and analyse our survey results, and to Bradley Tjandra for spending multiple working days (!) supporting the data analysis for this evaluation. Both Nathan and Bradley conducted this work as part of The Good Ancestors Project.

1. ^{^}

    Throughout the impact evaluation, unless otherwise specified, we are including our donation platform as part of GWWC. This includes the period before we acquired it from EA Funds in 2022. The reason we are considering it “GWWC” for that period is because the platform’s cost-effectiveness at the time is still relevant to its current and projected cost-effectiveness.

2. ^{^}

   All of our results are adjusted for inflation and reported in 2022-equivalent USD.

3. ^{^}

    We distinguish between the “funding we caused to go to effective charities from existing pledgers” and the “value we will have generated for effective charities from new pledges.” The former is purely retrospective (only looking at donations that have been made) whereas the latter also includes donations we expect will be made in the future. We do not add these two estimates together, but take a weighted average of the two to avoid double-counting our impact.

^{^}

Specifically, we had 4.5x as much impact if we look at the value of the GWWC Pledge in terms of the number of new pledges taken in that period multiplied by our estimated value generated per Pledge. We had 1.5x as much impact if we instead just look at the amount of money existing GWWC Pledgers donated because of us in 2020–2022.

^{^}

This assumes causality from correlation, which we think is plausible in this case (we don’t assume this causality for our overall impact estimates).

^{^}

Note that these figures do not account for non-recorded donations and so could exaggerate how heavy-tailed the distribution is.

Show all footnotes