Effective Altruism Forum
EA Forum

Hide table of contents
Comment Permalink
kierangreig🔸5
0
0

Hi Michelle and Sam,

Thanks so much for providing such detailed answers to my questions :). I think that I am getting a much better idea now. Thanks Sam for linking me to the comments of GWWC’s fundraising prospectus- the discussions there cover most of my questions. If it’s not too much to ask I would really like it if you answered a few more questions that I have.

Just to follow up on Sam’s answer to my first and second questions.

To help me understand how I should feel about the ratio GWWC uses for actual donations to pledged donations could you please outline what percentage of members record enough information in My Giving for their information to be used in calculating the ratio of actual donations to pledged donations that was used in GWWC’s most recent realistic impact estimate?

To clarify, are you talking about the impact of changes to members' income over time, or asking whether we're accounting for potential changes to donation patterns over time which affect the counterfactual ratio?”

I was talking about accounting for potential changes to donation patterns over time which affect the counterfactual ratio- sorry that wasn’t clear :). I certainly agree that it’s really hard to know how accurate people’s counterfactual estimates are. I wonder what the best way of doing estimates for counterfactuals is. Do you guys think that it might be more accurate to ask people each year about their counterfactual estimates rather than asking people once and then using that number for the next 40 years to calculate the counterfactual impact of GWWC?

One last question. GWWC seems to be the only EA fundraising organization I know of which calculates its potential impact for 40 years into the future. Do you guys think that other EA fundraising organizations like Charity Science and Raising for Effective Giving should attempt to do this type of realistic impact calculation for 40 years into the future so that donors who are trying to decide between these organizations may be able to more accurately compare the different organizations?

On a sidenote, I think that up and down votes of comments provide a really valuable source of feedback. As a result I am interested if people have ideas about the possible reasoning behind the down vote or down votes of my previous comment so that I can hopefully improve my future comments :).

Reply
Benjamin_Todd2
0
0

GWWC seems to be the only EA fundraising organization I know of which calculates its potential impact for 40 years into the future.

Even if you only focus on donations that have already been made, and ignore pledges, GWWC has a high positive multiplier. Moreover, completely ignoring the future value of pledges would be really pessimistic.

I think REG should try to value future pledged donations.

With CS you should value the future value of legacy commitments, even though they will take 20-60 years (though you'd need to apply discounting). (Presumably you... (read more)

Reply
3
Michelle_Hutchinson
I’m glad you feel you’re getting a proper sense! It’s surprisingly hard to describe all the information in an informative and intelligible way. I’m afraid I have no idea why the comment was downvoted. Given the number of upvotes, it could have just been a mistake? The calculation we used was for people who joined before 2013, and it was 137 out of 268 people, or 51%. Updating it using the 2013 and 2014 cohorts didn’t make much difference (a very slight increase in the percentage), so we continued using the same figure. The cohort data linked to above gives an indication of how many members for 2013 and 2014 gave us enough information to calculate a pledge percentage. With regard to the counterfactuals, we did at one point in the past ask that. People felt their answers weren’t more accurate when given later on though. The thing that seemed to be going on was that while when you’re considering joining but haven’t yet you have a fairly good sense of how you might act if you join, and if you don’t, but the longer you’re a member the less you identify with the counterfactual of not having joined and so the weirder it feels to think about what would have happened if it had been true. Asking people about these counterfactuals makes it harder to get other information we need (like income and giving data, and updated professions), either because we put it on the same email/survey making people more likely to put off responding to that, or because we send out an additional email/survey, increasing people’s fatigue of being asked stuff by us. So overall asking this seemed to make our calculations worse rather than better. We’re actually not the only EA org who tries to calculate our overall impact rather than just value which has already come to fruition – for example, 80,000 Hours also does. This calculation is actually very different from one which would try to capture our ‘potential impact for 40 years into the future’. If we ceased all our activities today, I would ex
See in context
by Michelle_Hutchinson
1 min read 34

11

Giving What We Can
Frontpage

 

 

Giving What We Can is a community of effective givers – an international society that helps people commit to giving more, and giving more effectively. Our members donate at least 10% of their career incomes to the charities they believe will help others the most.

Over the last year, our community has doubled in size. Our members have now collectively donated more than $10 million, and pledged over half a billion dollars! We have been testing out a range of ways to attract and retain new members, and seem to have found some to be really successful. Amongst these are setting up new local chapters and engaging in individual outreach. We would like to capitalise on this by scaling up some of the most promising avenues for attracting and retaining new members. To do so, we need to increase our team size. We've been lucky enough to have had really excellent applicants in our recent recruitment round. In order to carry out our plans — detailed in our fundraising prospectus — we need to reach our fundraising goal.

We think that the most relevant considerations for choosing whether to donate to us are our strong track record, and our plans for growth. But we also regularly calculate our effectiveness, to ensure that we are providing a substantial leverage ratio for our top charities. Our most up-to-date estimate for this indicates that for every $1 spent by Giving We Can, around $103 (counterfactually adjusted and time-discounted) will be moved to top charities.

Our budget for 2016 is £475,000, of which we already have £193,000 pledged. We therefore need to raise a further £282,000. You can find a full breakdown of our budget, along with details on our plans and our cost-effectiveness estimates on our fundraising page — you’ll also find a link with instructions for how you can donate. If you have any questions, please get in touch with me at michelle.hutchinson@givingwhatwecan.org.


If you decide to support us, I’d like to extend a huge thanks on behalf of the whole team — your generosity means that we can continue to help make the world a better place as effectively as possible.

11

0
0

Reactions

0
0
Comments34
Sorted by
New & upvoted
Click to highlight new comments since:
kierangreig🔸10
0
0

Hi Michelle,

Thank you so much for answering questions like this. I think it’s really worthwhile :). When you have a free moment it would be great if you could answer these questions. Unlike Peter I have only 6 questions:

1.) Should GWWC’s realistic impact estimate include the % of people each year who fulfill their pledge? For instance, the summary of the 2014 Giving Review states that 20-55% of people who pledged from 2009-2013 didn’t fulfill their pledge in 2014.

2.) On what information is the ratio of actual donations to pledged donations used in GWWC’s most recent realistic impact estimate based upon?

3.) Is the current technique that GWWC uses to calculate counterfactuals more likely to overestimate or underestimate GWWC’s impact given that the counterfactual percentage of all future donations is estimated when people initially take the pledge rather than when they make their donations in subsequent years?

4.) In Nick Beckstead’s 2013 quantitative performance review of GWWC he noted:

it is very unclear to what extent the new members are a result of the activities of GWWC’s staff, rather than organic growth.

What is your response to this?

5.) Around 6 months ago GWWC’s realistic impact estimate was a ratio of 60:1 and it’s now 104:1. Why has this number changed and do you expect it to change that much in another 6 months?

6.) It seems that GWWC’s comparative advantage is in generating and maintaining pledges for effective charities. Given that there is a large overlap between GWWC’s and GiveWell’s charity recommendations and GiveWell being in a better position to continue charity evaluations, why is it worth GWWC continuing charity evaluations?

Reply
Benjamin_Todd5
0
0

Also on Nick Beckstead's review, the reason he made that comment is that in 2013, it looked like growth was linear (2011 and 2012 both added 100 members), which made it more unclear whether marginal activities were helping.

The situation is different today - growth is definitely faster than linear.

There's now been two new impact evaluations after Nick wrote his, with much more in-depth and up-to-date figures.

Reply
SamDeere5
0
0

Hi Kieran,

Michelle is in a better position to answer some of these, but I'll answer the ones I can. I'd also suggest having a look at the comments section of our last fundraising prospectus, which covered some similar ground and which may provide more detail to some of your questions.

1) This is largely covered in the step Accounting for members donating a different amount than they pledged, which uses data from members who have reported their donations in My Giving, and comparing their actual donations with their pledges. The upper bound estimate in the Giving Review (80% of people keeping their pledge) uses the same dataset, but only takes into account a binary 'pledge met' vs. 'pledge not met'. The ratio of pledges to donations (117%) has more bearing our calculations because it captures both people who donate less than they pledge, and people who donate substantially more. Overall, due to people on average donating more than their pledge, the ratio is actually larger than 1:1 (so, even if only 80% of members hit their pledge amount, the number of people donating more than their pledge means the cohort as a whole donates more than it pledges).

The only quibble that you might have here is whether this cohort (people who report donations in My Giving) donates at a substantially different rate than people who don't report their donations (after we've factored out people who have both gone silent, and stopped donating, as per the earlier Accounting for membership attrition step). We have good reason to think this isn't the case (we know a lot of people personally who choose not to use My Giving, but who keep their pledges), but if you were more pessimistic about this, you could downweight the Ratio of Actual Donations to Pledged Donations in the spreadsheet (currently 1.17).

2) As above, this was calculated using data from members who have reported their donations in My Giving, and taking the ratio between their pledged amount and their reported donations, averaging over all members. See this section of the impact page for more info.

3) To clarify, are you talking about the impact of changes to members' income over time, or asking whether we're accounting for potential changes to donation patterns over time which affect the counterfactual ratio?

We're currently calculating our counterfactuals based on the amounts members say they would have donated without us — we haven't modelled behaviour changes into the future, and I'm not sure what we'd base such a model on. Whether it's conservative or optimistic is unclear, but I'd say that this is probably a wash — it's hard to know whether people's predictions of what they would have donated are overall optimistic or pessimistic. In our conversations with members, many people who say that they would have donated 10% without us also tell us that we're a useful commitment device (indicating that perhaps they wouldn't stick to their 10% without us, and that our counterfactual impact is actually greater than what we've accounted for in the model).

If you're just talking about the effect of members' income on the counterfactuals (because the calculations assume they will be static, when in reality income is likely to rise) then we think the calculation is fairly conservative. See the Donations Pledged By Members section of the impact page:

This methodology relies on the accuracy of members’ predictions about their future income. In general we have found that these predictions seem conservative, as most people underestimate their future earning potential[7] . If members do not estimate their future salary, we use the median salary for their country. We think that is a fairly pessimistic assumption, as our median member has an expected earning potential higher than the median wage[8] .

Footnotes 7 and 8 expand on this:

  1. For example, many members estimate their future income will be the same as their current income, even though they are at the beginning of their careers - in reality, income typically increases throughout a person’s career

  2. For example, many members attend prestigious universities and/or are pursuing careers that have an average salary much higher than the median wage

See also this comment made on the last prospectus for some discussion of the effect of modelling changing income over time. Using the same model with the updated figures yields a ratio of between 69:1 (using a fairly arbitrary starting pledged amount of $4,200,000 which produces donations over members' careers equivalent to the $344 million pledged amount, but accounts for income growth) and 157:1 (assuming that the current pledges correspond to current income levels, and that all members are at the beginning of their careers). You can play with this assumption by editing the figure in cell C2 on the 'Calculations' sheet of this spreadsheet, and the income growth rates at the bottom of the column.

5) We've taken into account an additional year (2014), where we had strong growth, but where our costs were not significantly higher. Our membership more than doubled (386 in 2009-13 vs 792 in 2009-14) but our costs only went up by around 40% (£238k in 2009-13 vs £332k in 2009-14). The assumptions have remained essentially the same, so the difference in those ratios accounts for most of the difference (with a less significant amount being due to small changes in membership attrition and counterfactual pledge ratios).

As we note in the caveats, we do expect this amount to go down in future as our staff costs increase, and we don't want people to fixate on it as a predictor of our impact. We see it more as a sanity check of whether we're a good bet, vs giving money to other effective causes.

The degree to which this will change significantly in future really depends on how strong our member growth vs. costs growth is. If the cost of creating a new member hits diminishing marginal returns soon (not at all unlikely), then it's likely to drop back fairly quickly. We don't see this as particularly troubling — so long as our absolute number of members keeps increasing and the ratio is positive, then we're still a good bet.

We doubled staff numbers over 2015 (3 > 6) and we're hiring again now (6 > 8 or 9, depending on fundraising), so it's likely that this will push it back down. We think that maximising future membership growth will be contingent on broadening the skillset of our team (and just having more hands on deck to do outreach work would be a huge help!). Expanding the team, strengthening our organisation, and increasing our growth rate seems very important right now (and much more important than maintaining this ratio at the current level). My guess is that it will settle somewhere between 20:1 and 60:1 — not quite as impressive as 104:1, but still suggestive that we're making a big difference!

(This difference is similar to the reason that we don't think that using "overhead" is a good measure of a charity's effectiveness. In effect, this is our overheads increasing, but so long as this leads to greater (counterfactual) overall member growth and donations to top charities, then we should be happy for the ratio to drop.)

6) Hauke, our Director of Research answers this question here. The short version:

  • Supplement GiveWell research and find new charities within our comparative area of advantage (global poverty reduction)
  • Independently check GiveWell recommendations and provide resilience to the effective charity evaluation system
  • Provide supervision for students/early career researchers who want to focus on effectiveness
  • Ensure in-house credibility when talking about top charities and fact-checking all of our public-facing material

Hope that helps!

Reply
Michelle_Hutchinson2
0
0

Hi Kieran,

Glad it’s useful!

Sam’s covered most of what I would say, but on 4): I think he’s right that it’s difficult to know to what extent growth is down to particular actions of our staff versus other growth factors (although we do have a bunch more information than we did back then). There are a lot of different factors at play, and often there are a lot of necessary conditions for someone becoming a member. Often it’s difficult even to know the difference between ‘organic growth’ and ‘staff activities’ – does someone joining due in part to an action of one of our staff 3 years ago count as organic or staff? A couple of concrete examples: 1) the Cambridge pledge event last year seemed to get around 80 people to join who wouldn’t otherwise have joined. The pledge event wouldn’t have happened without the Cambridge chapter, which wouldn’t have started up without GWWC central. The central team also did various things to support the chapter, and built a website where it’s easy to join etc. But initiative was taken and work was done by a group of students acting mostly on their own. 2) Marinella has been writing to people who attended EA Global who weren’t yet members of GWWC – learning about how they found out about us, answering questions they have etc. 9 of the people she contact joined GWWC directly after / during their conversations with her. On the one hand, that seems concretely down to her reaching out. On the other hand, EA Global was also a necessary part. And on the other hand again, the GWWC staff were at EA Global and actively making people aware of GWWC and effective giving there.

Individual outreach of the kind Marinella has been doing, along with setting up chapters from scratch using chapter seeding, seems quite clearly to be attributable to particular actions by particular of our staff. But it's important to focus on only those kinds of really direct things. Advice from marketing seems to indicate that if you want to get people on board it’s important to have a strong brand, look professional etc. It would be very surprising if that wasn’t important for us too. That kind of thing isn’t really possible from organic growth. But it’s difficult to know precisely what effect it has – all we can do is learn as much as possible about what does and doesn’t work for other organisations, try things out, and see how it seems to affect our growth.

Reply
kierangreig🔸5
0
0

Hi Michelle and Sam,

Thanks so much for providing such detailed answers to my questions :). I think that I am getting a much better idea now. Thanks Sam for linking me to the comments of GWWC’s fundraising prospectus- the discussions there cover most of my questions. If it’s not too much to ask I would really like it if you answered a few more questions that I have.

Just to follow up on Sam’s answer to my first and second questions.

To help me understand how I should feel about the ratio GWWC uses for actual donations to pledged donations could you please outline what percentage of members record enough information in My Giving for their information to be used in calculating the ratio of actual donations to pledged donations that was used in GWWC’s most recent realistic impact estimate?

To clarify, are you talking about the impact of changes to members' income over time, or asking whether we're accounting for potential changes to donation patterns over time which affect the counterfactual ratio?”

I was talking about accounting for potential changes to donation patterns over time which affect the counterfactual ratio- sorry that wasn’t clear :). I certainly agree that it’s really hard to know how accurate people’s counterfactual estimates are. I wonder what the best way of doing estimates for counterfactuals is. Do you guys think that it might be more accurate to ask people each year about their counterfactual estimates rather than asking people once and then using that number for the next 40 years to calculate the counterfactual impact of GWWC?

One last question. GWWC seems to be the only EA fundraising organization I know of which calculates its potential impact for 40 years into the future. Do you guys think that other EA fundraising organizations like Charity Science and Raising for Effective Giving should attempt to do this type of realistic impact calculation for 40 years into the future so that donors who are trying to decide between these organizations may be able to more accurately compare the different organizations?

On a sidenote, I think that up and down votes of comments provide a really valuable source of feedback. As a result I am interested if people have ideas about the possible reasoning behind the down vote or down votes of my previous comment so that I can hopefully improve my future comments :).

Reply
Michelle_Hutchinson3
0
0

I’m glad you feel you’re getting a proper sense! It’s surprisingly hard to describe all the information in an informative and intelligible way.

I’m afraid I have no idea why the comment was downvoted. Given the number of upvotes, it could have just been a mistake?

The calculation we used was for people who joined before 2013, and it was 137 out of 268 people, or 51%. Updating it using the 2013 and 2014 cohorts didn’t make much difference (a very slight increase in the percentage), so we continued using the same figure. The cohort data linked to above gives an indication of how many members for 2013 and 2014 gave us enough information to calculate a pledge percentage.

With regard to the counterfactuals, we did at one point in the past ask that. People felt their answers weren’t more accurate when given later on though. The thing that seemed to be going on was that while when you’re considering joining but haven’t yet you have a fairly good sense of how you might act if you join, and if you don’t, but the longer you’re a member the less you identify with the counterfactual of not having joined and so the weirder it feels to think about what would have happened if it had been true. Asking people about these counterfactuals makes it harder to get other information we need (like income and giving data, and updated professions), either because we put it on the same email/survey making people more likely to put off responding to that, or because we send out an additional email/survey, increasing people’s fatigue of being asked stuff by us. So overall asking this seemed to make our calculations worse rather than better.

We’re actually not the only EA org who tries to calculate our overall impact rather than just value which has already come to fruition – for example, 80,000 Hours also does. This calculation is actually very different from one which would try to capture our ‘potential impact for 40 years into the future’. If we ceased all our activities today, I would expect that quite a few people would join and donate due to our past activities. I also expect that we’ve gained quite a bit of experience which will make us more effective in future than we have been in the past. So this calculation doesn’t even capture the potential impact of all the activities we’ve undertaken so far, let alone our potential impact into the future. Yes, I think it’s very important for EA organisations to try to get a comprehensive picture of their overall impact, both for internal use and for informing others.

Reply
kierangreig🔸3
0
0

Thanks again for answering these questions Michelle and Sam :)

Reply
Benjamin_Todd2
0
0

GWWC seems to be the only EA fundraising organization I know of which calculates its potential impact for 40 years into the future.

Even if you only focus on donations that have already been made, and ignore pledges, GWWC has a high positive multiplier. Moreover, completely ignoring the future value of pledges would be really pessimistic.

I think REG should try to value future pledged donations.

With CS you should value the future value of legacy commitments, even though they will take 20-60 years (though you'd need to apply discounting). (Presumably you do give them positive value even though they're a long way in the future? :)) The case for valuing future birthday fundraisers is less strong because there's no analogous long-term commitment.

Moreover, in all cases, we shouldn't get too hung up on historical impact. Most of our impact lies in the future, so what really matters is the potential upside and probability of that upside. Historical money moved is only a weak indication of that.

https://80000hours.org/2015/11/take-the-growth-approach-to-evaluating-startup-non-profits-not-the-marginal-approach/

Reply
kierangreig🔸0
0
0

Hi Ben,

Thanks for responding :)

Even if you only focus on donations that have already been made, and ignore pledges, GWWC has a high positive multiplier. Moreover, completely ignoring the future value of pledges would be really pessimistic.

I totally agree and think that a really interesting question is what the future value of pledges should be. I think may also be worth mentioning that if we focus on donations that have already been made, my understanding is GWWC’s impact is an order of magnitude less than their current realistic impact estimate. I am not sure how exactly we should weigh that information.

With CS you should value the future value of legacy commitments, even though they will take 20-60 years (though you'd need to apply discounting). (Presumably you do give them positive value even though they're a long way in the future? :))

At CS we sure do value the future value of legacy commitments :). We haven’t yet determined exactly how we will calculate their expected value.

Reply
Benjamin_Todd1
0
0

How do should you weigh the information?

Zooming out for a bit, if you're trying to evaluate GWWC's historical impact, there's five categories of impact:

  1. Money pledgers have already donated
  2. Money pledgers will donate in the future
  3. People who will take the pledge in the future due to past investments (e.g. GWWC's website will keep creating pledgers even if they closed tomorrow).
  4. Money influenced by people who didn't take the pledge
  5. Impact that isn't in the form of donations to top charities (creating the EA movement, founding CEA, their research contributions, spinning off 80k, substantial policy influence)

GWWC are very modest and only focus on (1) and (2) :)

To evaluate the total impact, you need to calculate the expected value for each component. You should make your own estimates given the best information you have.

Only counting (1) would be a very poor estimate. (2) to (5) are very unlikely to be zero. My personal guess is that:

(2) is about 10x (1) [because I think GWWC's internal estimates are reasonable] (3) and (4) are similar to (1) (5) is much larger than (2)

However, even if you only cared about (1), GWWC's multiplier would still be about 10x, which would make them more cost-effective than GiveWell recommended charities, and equal to or more cost-effective than the other EA fundraising orgs (in the past). So I'm not sure it's even that decision relevant.

Reply
Benjamin_Todd2
0
0

Personally if GWWC central closed tomorrow, I expect most of the growth would stop. There would be a remaining constant trickle of members from the website (providing at most linear growth, whereas GWWC has been growing faster than linear, and most members don't join just from reading the website alone). Initially there would be continued growth from the chapters too, but the numbers would start to shrink. You may get continued growth from the EA community promoting GWWC, but I expect it would be much smaller.

More concretely, GWWC staff right now pursue several channels to growth that wouldn't happen without them:

  1. Speaking to interested people one-on-one (few join just from written content alone; being made 'the ask' is really important)
  2. Fostering chapters (e.g. giving them materials, arranging pledge drives, encouraging them, doing events) - most of this wouldn't happen otherwise.
  3. Seeking press attention to get new people involved - ditto, mostly wouldn't happen otherwise.

And there's lots of other things they could do at the margin to get even more growth e.g. boosting the conversion rate of the website.

Finally, like Michelle says, much of the continued growth there would be if GWWC closed is due to staff efforts in past years (e.g. developing chapters in the first place), so should be included at some point in your evaluation. It would be wrong to just ignore new members from this year who didn't result from activities by staff this year. Future staff efforts can also be expected to generate members many years into the future.

In this sense, GWWC leverage ratio is a big underestimate of their effectiveness. Past efforts will actually generate a continued stream of members who aren't being counted.

Reply
number423
0
0

More concretely, GWWC staff right now pursue several channels to growth that wouldn't happen without them:

Speaking to interested people one-on-one (few join just from written content alone; being made 'the ask' is really important) Fostering chapters (e.g. giving them materials, arranging pledge drives, encouraging them, doing events) - most of this wouldn't happen otherwise. Seeking press attention to get new people involved - ditto, mostly wouldn't happen otherwise.

Does GWWC have rough estimates of how many members each of these channels lead to? Or could you or other CEA staff hazard a guess, even rough?

Reply
Dale10
0
0

Hey Michelle,

In the past we've discussed how desirable it would be for GWWC to release cohort data to allow potential donors to properly evaluate how much value GWWC creates. Without it its hard for us to estimate the lifetime value of new members. While it seems clear GWWC is positive value, we need to be able to compare it to other effective charities. At the time it was suggested that you would release this data; any chance we could see it in time for this giving season - or if not, in the new year?

Reply
Michelle_Hutchinson7
0
0

Hey Dale, Here's our most recent giving review, which has that information in. The section you want is 1 d. 'Amounts donated'. We didn't end up including it in the webpages on plans and effectiveness because it's captured in our calculation in 'rate of leaving, going silent and not giving' (in the 'realistic calculation' flow chart), and we didn't want to make the overall too long and complex. There are some other sections in there you might be interested in too, like trust statistics, pledge fulfilment and giving, and employment status.

Reply
aliwoodman6
0
0

Hi, Just to add to this, here is a link to a document I made with the sort of data you suggested sharing in the conversation earlier this year about cohort data. (Some of this is already contained in the Giving Review but I've presented it by cohort to hopefully make it easy to read). I agree that it's useful to collect and share, so thank you for the prompt!

Reply
Benjamin_Todd2
0
0

This is impressive - several thousand dollars of donation per member, with no obvious trend downwards excluding the first cohort (which was filled with especially large givers). This easily suggests a member is worth tens of thousands of dollars of donations.

And I'd expect My Giving data to underreport, since there will be people who donate but don't both to fill it in.

Reply
Peter Wildeford0
0
0

Thanks Allison for such a clear document!

Do you have similar numbers for the 2013 pledge?

Reply
aliwoodman1
0
0

Hi Peter, I don't yet but am working on it: I'll post them here when they're done :)

Reply
aliwoodman3
0
0

Here is the cohort information for 2013

Reply
Peter Wildeford1
0
0

Do you know what the overlap is like? Are the people going silent in 2013 the same people who go silent in 2014?

Reply
aliwoodman2
0
0

Hi again! I've now had a look at this - for the cohorts who joined before 2013 it looks like 30% of those who reported meeting their pledge in 2014 had not reported meeting their pledge in 2013. So this shows that just because someone didn't report their donations in 2013 doesn't mean they didn't the following year. I have added a breakdown of this in the 2014 cohort document. (NB I also made some very small corrections to the 2013 cohort info, which slightly increased the numbers for 'people who recorded some income and donation data' and changed the median %'s donated)

Reply
Peter Wildeford9
0
0

Hey Michelle,

Thank you very much for sharing the cohort analysis. I find it very valuable to understanding GWWC's membership flow. I also know it's stressful to release the data and have it publicly scrutinized, especially during a high-stakes fundraising round. But I think it's very important to have such public scrutiny when requesting public funds. :)

I have seven questions:

1.) I see from the Giving Review that GWWC had 386 members who joined prior to 2014, with 186 "going silent" (not responding with income and donation numbers) and 43 not completing the pledge, for an overall 40.7% "knowably kept pledge" rate.

But in the Fundraising doc, the "Realistic calculation" has a rate of leavers of 0.017 and a rate of people going silent of 0.031, for an overall estimate of "knowably kept pledge" rate of 95.2%. Do you have more information on how the attrition rate was calculated in the "Realistic calculation" and why these two numbers are so different?

-

2.) You state a counterfactual donation rate of 51%. Can you explain where that number comes from?

-

3.) You say that 66% of people say you've affected their choice of charity. Is this 66% of the entire population, or 66% of the 49% of people who would have donated anyway?

-

4.) What does the "effectiveness ratio" of 0.5 mean and how was it calculated?

-

5.) What credence, if any, do you give to the thought that the outside view for pledging working so successfully is pretty weak? My impression is that fundraising doesn't make much use of pledges, hough they do frequently seek out "monthly donations" over one-time giving). If it were more successful, would other orgs do it? (Of course, the inside view could still be strong and it could be that GWWC has caught on to something innovative that other people just haven't seen yet. And I suppose other orgs like REG have also had a lot of success with pledges.)

-

6.) This is a positive question where I think GWWC underestimates it's impact -- what credence do you give to Ben Todd's position that EA is underemphasizing potential for growth and overestimating historical money moved metrics? Do you think GWWC is in a position to try new things it hasn't before? What kind of learning value do you think is in that? Do you think GWWC will have more economies of scale in getting new pledgers as it grows?

-

7.) I hate to move the goalposts after releasing the cohort analysis, but I think what we ideally would like is "survival analysis". I find it interesting to compare how likely a GWWC pledger is to keep a pledge to how likely a person taking out a loan is to pay back their loan (my job is to calculate that second one). One tool we use is survival analysis, which is kind of like an improved cohort analysis, where we make predictions like given that someone paid back in year 1, how likely are they to pay back in year 2, etc.? My guess is that we could do something very similar for GWWC members and I'd be happy to help do that when my time allows (maybe sometime early next year).

I'd also be really interested in using any data we have on members (e.g., demographics, income, how they heard about GWWC, level of contact with GWWC, etc.) to help predict which members are more likely to keep their pledge.

Reply
Michelle_Hutchinson5
0
0

Hi Peter,

Thanks for your questions. As you say, it's very important to make sure people have a really thorough understanding of what's going on, both with their money, and in general in the sphere of helping others, so that we can build on each other's work. You might like to check out our impact page, which has more detail on the methodology of our calculations.

1) Silent members in the fundraising document are defined as members who we have not had any contact with/information from in at least 2 years i.e. people who fulfil the following: joined before 2013, have not filled in My Giving in at least 2 years, not known to staff at CEA personally, has not joined the online GWWC community facebook group (set up at the start of 2014), has not responded to any emails in at least 2 years. There are therefore relatively few of them compared with the number of members who we couldn't persuade to fill out My Giving in a given year. It's worth noting that while you might think that filling in My Giving is a small thing compared to giving 10% of your income, we know many people personally who do donate at least 10% but can't be persuaded to fill in My Giving - many people just find recording donations in any way a big hassle. The claim in the fundraising document is not that 95.2% of members will donate their pledge each year - it's more realistically estimating the % of members who will stop donating per year, who will not resume donating in future years.

2) From looking at the % members pledged and subtracting the percentage members say they would have given anyway, and dividing by the percentage pledged for each member, then averaging over these. More info on the Impact page of the website: https://www.givingwhatwecan.org/impact/#realistic-impact-calculation-1031

3) 66% of the entire population

4) This represents how much more effective the charities people give to are than the ones they would have given to otherwise, for the case where people say we affected their donation. It's simply an estimate, based on the reasoning that we're probably if we affected someone's donation it will be by nudging them to give in the developing world rather than developed world, or by nudging them from a typical developing world charity to one of our recommended charities. I therefore estimated that on average it might be something like a doubling in impact. That seems like a pretty conservative assumption to me.

5) I wouldn't say there are no other examples of it - tithing works quite like this, and there are various examples of people encouraging each other to 'promise' to uphold some value that's deemed important (I remember learning about the 'silver ring thing' in the US when I was at school). I think the general principles of banding together to support each other upholding your values are fairly well established, and the outside view seems to be positive about the general psychological principles underlying pledging and forming a community. Most fundraisers are fundraising for one particular charity, and it doesn't seem terribly surprising that pledges would seem less appealing if someone's asking you just to pledge to their organisation. I also think that we shouldn't be thinking of taking the pledge as the only thing - that's the measurable thing, and the time when we can ask people the most easily about how we affect their donations. But in practice, the reason people pledge is to do with the overall community, with the information we can provide etc. So we shouldn't think of it as 'you put a pledge page online and then lots of behaviour changes' which would be really surprising, but in a more holistic way. Overall, I think it is somewhat surprising how the concept seems to be working. But my surprise is more with how many people are willing to actually give 10% of their income to effective charities, than with the fact that there being other people around inspiring them and holding them to account would have a big effect on that willingness.

6) Yes, I basically agree with Ben Todd on that, that's why we tried to emphasise in our prospectus the importance of looking at our plan for growth, rather than the leverage numbers. We're always trying new ways of reaching out to people, changing the website to improve flow through and test different messages etc. I think it's important we have credibility as a stable organisation in terms of the pledge remaining the same, and continuing to have regular contact with members. But when it comes to trying out different ways of outreach, we can be continuously improving. Yes, I think there will be economies of scale. Eg. getting really good chapter resources made and online is high intensity, but can then be used by however many chapters we have. Last year we made a marginal assessment for members, but we were trying to focus more on our plans this year than on straightforward leverage ratios, so didn't include that calculation.

7) Sounds good, I look forward to hearing from you when you have time for it.

Reply
Gregory Lewis🔸3
0
0

Happy to help with the analysis project alluded to above.

Reply
Peter Wildeford1
0
0

Sounds great -- maybe we could collaborate. :)

Reply
Peter Wildeford0
0
0

P.S. One of my friends brings up a good point to me that ideally you want dollar-weighted attrition, where you look at the total amount of money donated in year N divided by the total amount of money pledged in year N-1 for year N. It's possible this number could be above 100% if people underestimate how much they'll donate.

Reply
Peter Wildeford1
0
0

P.P.S. Though for understanding a model of how GWWC members come and go, regular attrition is the preferred metric, though you may want to have a slight bias towards looking at employed members.

Reply
Josh Jacobson7
0
0

A big upvote for the quality of the comments from donors and GWWC here. I'd love to see this same level of inspection / data release / discussion throughout EA donor decisions.

Reply
Nekoinentr6
0
0

103:1 is an incredible fundraising ratio - how are you able to convince people to donate so much with such a small investment, apparently just a small amount of staff time, and how would others go about replicating this? If people could replicate your methods around the world it'd be highly desirable for them to do so.

Reply
Michelle_Hutchinson4
0
0

Hi Nekoinentr, I think unfortunately the answer isn't in the form of a simple, easily replicable method. It's surprisingly tricky and complex to figure out the various things that lead to people joining and donating. Our activities are described here. At the moment, one of the outreach methods we're finding most effective and scalable is setting up and supporting chapters. If you'd be interested in getting involved in with then, Jon would love to hear from you!

Reply
pererik3
0
0

Just made my last donation of the year to meet my GWWC pledge to give 10% of my income, $3184. I'm happy to be donating to CEA. This year I looked for a way to multiply the impact of my donation and giving to CEA/GWWC seemed like the best way to do this. On their most conservative estimate, they move $6 to a highly cost-effective charity for every $1 they spend. I can't think of a better way to spend my money!

Reply
SamDeere3
0
0

I've updated our impact page to include a spreadsheet that you can use to test our Realistic impact calculation. Find it under the "Spreadsheet" header.

Also, preparing this spreadsheet for public use revealed a minor error in our workings (the original spreadsheet was using an out-of-date figure for Proportion of people who say we've affected their choice of charity) — this has been corrected, and the impact ratio has accordingly shifted slightly from 103:1 to 104:1.

Reply
More from Michelle_Hutchinson
View more
Curated and popular this week
Relevant opportunities
View more