Also on Nick Beckstead's review, the reason he made that comment is that in 2013, it looked like growth was linear (2011 and 2012 both added 100 members), which made it more unclear whether marginal activities were helping.

The situation is different today - growth is definitely faster than linear.

There's now been two new impact evaluations after Nick wrote his, with much more in-depth and up-to-date figures.

Hi Kieran,

Michelle is in a better position to answer some of these, but I'll answer the ones I can. I'd also suggest having a look at the comments section of our last fundraising prospectus, which covered some similar ground and which may provide more detail to some of your questions.

1) This is largely covered in the step Accounting for members donating a different amount than they pledged, which uses data from members who have reported their donations in My Giving, and comparing their actual donations with their pledges. The upper bound estimate in the Giving Review (80% of people keeping their pledge) uses the same dataset, but only takes into account a binary 'pledge met' vs. 'pledge not met'. The ratio of pledges to donations (117%) has more bearing our calculations because it captures both people who donate less than they pledge, and people who donate substantially more. Overall, due to people on average donating more than their pledge, the ratio is actually larger than 1:1 (so, even if only 80% of members hit their pledge amount, the number of people donating more than their pledge means the cohort as a whole donates more than it pledges).

The only quibble that you might have here is whether this cohort (people who report donations in My Giving) donates at a substantially different rate than people who don't report their donations (after we've factored out people who have both gone silent, and stopped donating, as per the earlier Accounting for membership attrition step). We have good reason to think this isn't the case (we know a lot of people personally who choose not to use My Giving, but who keep their pledges), but if you were more pessimistic about this, you could downweight the Ratio of Actual Donations to Pledged Donations in the spreadsheet (currently 1.17).

2) As above, this was calculated using data from members who have reported their donations in My Giving, and taking the ratio between their pledged amount and their reported donations, averaging over all members. See this section of the impact page for more info.

3) To clarify, are you talking about the impact of changes to members' income over time, or asking whether we're accounting for potential changes to donation patterns over time which affect the counterfactual ratio?

We're currently calculating our counterfactuals based on the amounts members say they would have donated without us — we haven't modelled behaviour changes into the future, and I'm not sure what we'd base such a model on. Whether it's conservative or optimistic is unclear, but I'd say that this is probably a wash — it's hard to know whether people's predictions of what they would have donated are overall optimistic or pessimistic. In our conversations with members, many people who say that they would have donated 10% without us also tell us that we're a useful commitment device (indicating that perhaps they wouldn't stick to their 10% without us, and that our counterfactual impact is actually greater than what we've accounted for in the model).

If you're just talking about the effect of members' income on the counterfactuals (because the calculations assume they will be static, when in reality income is likely to rise) then we think the calculation is fairly conservative. See the Donations Pledged By Members section of the impact page:

This methodology relies on the accuracy of members’ predictions about their future income. In general we have found that these predictions seem conservative, as most people underestimate their future earning potential[7] . If members do not estimate their future salary, we use the median salary for their country. We think that is a fairly pessimistic assumption, as our median member has an expected earning potential higher than the median wage[8] .

Footnotes 7 and 8 expand on this:

1. For example, many members estimate their future income will be the same as their current income, even though they are at the beginning of their careers - in reality, income typically increases throughout a person’s career

2. For example, many members attend prestigious universities and/or are pursuing careers that have an average salary much higher than the median wage

See also this comment made on the last prospectus for some discussion of the effect of modelling changing income over time. Using the same model with the updated figures yields a ratio of between 69:1 (using a fairly arbitrary starting pledged amount of $4,200,000 which produces donations over members' careers equivalent to the $344 million pledged amount, but accounts for income growth) and 157:1 (assuming that the current pledges correspond to current income levels, and that all members are at the beginning of their careers). You can play with this assumption by editing the figure in cell C2 on the 'Calculations' sheet of this spreadsheet, and the income growth rates at the bottom of the column.

5) We've taken into account an additional year (2014), where we had strong growth, but where our costs were not significantly higher. Our membership more than doubled (386 in 2009-13 vs 792 in 2009-14) but our costs only went up by around 40% (£238k in 2009-13 vs £332k in 2009-14). The assumptions have remained essentially the same, so the difference in those ratios accounts for most of the difference (with a less significant amount being due to small changes in membership attrition and counterfactual pledge ratios).

As we note in the caveats, we do expect this amount to go down in future as our staff costs increase, and we don't want people to fixate on it as a predictor of our impact. We see it more as a sanity check of whether we're a good bet, vs giving money to other effective causes.

The degree to which this will change significantly in future really depends on how strong our member growth vs. costs growth is. If the cost of creating a new member hits diminishing marginal returns soon (not at all unlikely), then it's likely to drop back fairly quickly. We don't see this as particularly troubling — so long as our absolute number of members keeps increasing and the ratio is positive, then we're still a good bet.

We doubled staff numbers over 2015 (3 > 6) and we're hiring again now (6 > 8 or 9, depending on fundraising), so it's likely that this will push it back down. We think that maximising future membership growth will be contingent on broadening the skillset of our team (and just having more hands on deck to do outreach work would be a huge help!). Expanding the team, strengthening our organisation, and increasing our growth rate seems very important right now (and much more important than maintaining this ratio at the current level). My guess is that it will settle somewhere between 20:1 and 60:1 — not quite as impressive as 104:1, but still suggestive that we're making a big difference!

(This difference is similar to the reason that we don't think that using "overhead" is a good measure of a charity's effectiveness. In effect, this is our overheads increasing, but so long as this leads to greater (counterfactual) overall member growth and donations to top charities, then we should be happy for the ratio to drop.)

6) Hauke, our Director of Research answers this question here. The short version:

• Supplement GiveWell research and find new charities within our comparative area of advantage (global poverty reduction)
• Independently check GiveWell recommendations and provide resilience to the effective charity evaluation system
• Provide supervision for students/early career researchers who want to focus on effectiveness
• Ensure in-house credibility when talking about top charities and fact-checking all of our public-facing material

Hope that helps!

Hey Dale, Here's our most recent giving review, which has that information in. The section you want is 1 d. 'Amounts donated'. We didn't end up including it in the webpages on plans and effectiveness because it's captured in our calculation in 'rate of leaving, going silent and not giving' (in the 'realistic calculation' flow chart), and we didn't want to make the overall too long and complex. There are some other sections in there you might be interested in too, like trust statistics, pledge fulfilment and giving, and employment status.

Hi, Just to add to this, here is a link to a document I made with the sort of data you suggested sharing in the conversation earlier this year about cohort data. (Some of this is already contained in the Giving Review but I've presented it by cohort to hopefully make it easy to read). I agree that it's useful to collect and share, so thank you for the prompt!

Hi Peter,

Thanks for your questions. As you say, it's very important to make sure people have a really thorough understanding of what's going on, both with their money, and in general in the sphere of helping others, so that we can build on each other's work. You might like to check out our impact page, which has more detail on the methodology of our calculations.

1) Silent members in the fundraising document are defined as members who we have not had any contact with/information from in at least 2 years i.e. people who fulfil the following: joined before 2013, have not filled in My Giving in at least 2 years, not known to staff at CEA personally, has not joined the online GWWC community facebook group (set up at the start of 2014), has not responded to any emails in at least 2 years. There are therefore relatively few of them compared with the number of members who we couldn't persuade to fill out My Giving in a given year. It's worth noting that while you might think that filling in My Giving is a small thing compared to giving 10% of your income, we know many people personally who do donate at least 10% but can't be persuaded to fill in My Giving - many people just find recording donations in any way a big hassle. The claim in the fundraising document is not that 95.2% of members will donate their pledge each year - it's more realistically estimating the % of members who will stop donating per year, who will not resume donating in future years.

2) From looking at the % members pledged and subtracting the percentage members say they would have given anyway, and dividing by the percentage pledged for each member, then averaging over these. More info on the Impact page of the website: https://www.givingwhatwecan.org/impact/#realistic-impact-calculation-1031

3) 66% of the entire population

4) This represents how much more effective the charities people give to are than the ones they would have given to otherwise, for the case where people say we affected their donation. It's simply an estimate, based on the reasoning that we're probably if we affected someone's donation it will be by nudging them to give in the developing world rather than developed world, or by nudging them from a typical developing world charity to one of our recommended charities. I therefore estimated that on average it might be something like a doubling in impact. That seems like a pretty conservative assumption to me.

5) I wouldn't say there are no other examples of it - tithing works quite like this, and there are various examples of people encouraging each other to 'promise' to uphold some value that's deemed important (I remember learning about the 'silver ring thing' in the US when I was at school). I think the general principles of banding together to support each other upholding your values are fairly well established, and the outside view seems to be positive about the general psychological principles underlying pledging and forming a community. Most fundraisers are fundraising for one particular charity, and it doesn't seem terribly surprising that pledges would seem less appealing if someone's asking you just to pledge to their organisation. I also think that we shouldn't be thinking of taking the pledge as the only thing - that's the measurable thing, and the time when we can ask people the most easily about how we affect their donations. But in practice, the reason people pledge is to do with the overall community, with the information we can provide etc. So we shouldn't think of it as 'you put a pledge page online and then lots of behaviour changes' which would be really surprising, but in a more holistic way. Overall, I think it is somewhat surprising how the concept seems to be working. But my surprise is more with how many people are willing to actually give 10% of their income to effective charities, than with the fact that there being other people around inspiring them and holding them to account would have a big effect on that willingness.

6) Yes, I basically agree with Ben Todd on that, that's why we tried to emphasise in our prospectus the importance of looking at our plan for growth, rather than the leverage numbers. We're always trying new ways of reaching out to people, changing the website to improve flow through and test different messages etc. I think it's important we have credibility as a stable organisation in terms of the pledge remaining the same, and continuing to have regular contact with members. But when it comes to trying out different ways of outreach, we can be continuously improving. Yes, I think there will be economies of scale. Eg. getting really good chapter resources made and online is high intensity, but can then be used by however many chapters we have. Last year we made a marginal assessment for members, but we were trying to focus more on our plans this year than on straightforward leverage ratios, so didn't include that calculation.

7) Sounds good, I look forward to hearing from you when you have time for it.

Happy to help with the analysis project alluded to above.

P.S. One of my friends brings up a good point to me that ideally you want dollar-weighted attrition, where you look at the total amount of money donated in year N divided by the total amount of money pledged in year N-1 for year N. It's possible this number could be above 100% if people underestimate how much they'll donate.

Hi Nekoinentr, I think unfortunately the answer isn't in the form of a simple, easily replicable method. It's surprisingly tricky and complex to figure out the various things that lead to people joining and donating. Our activities are described here. At the moment, one of the outreach methods we're finding most effective and scalable is setting up and supporting chapters. If you'd be interested in getting involved in with then, Jon would love to hear from you!