Proposal for an Pre-registered Experiment in EA Outreach

by Linch 8th Jan 201710 comments


Can talking about GWWC for 90 minutes actually get somebody to take the Pledge?


Note: I timed myself so I will not take more than 30 mins on the first draft (and hopefully less than an hour on this post overall)


Surface analysis of Giving What We Can members talking about the pledge with friends suggest that this results in unusually high marginal value in terms of getting new pledges. In particular, there appears to be a conversion rate of roughly 10% (1 in 10 people you talk to go on to take the pledge within a month).


This has naturally led to some skepticism of such extraordinary claims. Which is very understandable.


Thus, in a comment earlier, I suggest doing a pre-registered outreach trial to rectify this.


Here is the procedure:

  1. We gather 5-101 people who:

    1. Have never directly talked to non-GWWC members about GWWC in the last 6 months

    2. Are willing to experiment with talking about Giving What We Can, and have spare time

    3. Are themselves Giving What We Can members or Try Givers


  1. The experimentees2 are known to each other.


  1. Each experimentee is instructed to contact3 a minimum of 5 friends and a maximum of 204 who have not taken the pledge (but the experimentee thinks should take the pledge) within five days after the experiment starts.


  1. Experimentees are free to use any of the GWWC/EA resources on convincing people, including talking to people very familiar with GWWC like myself.5 The one caveat is that they cannot ask CEA staff or core volunteers on who to contact. Also CEA should endeavor not to contact the same people within the experimental time period, though of course accidents might happen (and should be recorded).


  1. Experimentees take notes6 on how many people they’ve messaged, and how many people responded to the initial conversation, reactions, etc., and (arguably most importantly) how much time it has taken them.


  1. 20 days after the experiment began, we look at how many of the people contacted actually went on to take the pledge.


  1. The results of the study will be written up in the EAForum.


  1. If anybody’s interested, we can do a follow-up on the same people 1 year later to see how much they have donated.

Prediction: My 90% credible interval7 is that this experiment should have a 0-20% success rate8 (median 7%).


This experiment does not have a control group. I believe this is okay because the base rate of random individuals (including EAs) taking the pledge within any 15-day span should be fairly low.


If you’re interested in participating in the experiment, and fit condition 1), PLEASE contact me ASAP at email.linch[at]gmail[dot]com. Thanks so much!





1 The experiment will not start until we have at least five people. We’ll try to wait a bit to get 10 people.


2 Peter Hurford has volunteered to be one such experimentee.


3 Experimentees can use any format: FB PM, text, phone, email or face-to-face. The object to generalize is whether “people who have not previously talked about GWWC can, using the best of their intellect and understanding of personal circumstances, get others to take the pledge in the time allotted,” not “Does FB messenger work better or worse than irl conversations?”


4 Again, the exact number is up to individual discretion. Some people may only have 6-7 friends who are likely GWWC material, others may have over 15 who just need a slight push. Allowing experimentees this freedom is reasonable given the information we’re trying to generalize.


5 This is reasonable, again, because the same resources will be available to people outside of the experiment.


6 Experimentees should at least try to coordinate on not messaging the same people. A few doubles is okay however as long as the final analysis doesn’t double-count impact.


7 This is somewhat contingent on how many people end up being contacted.I reserve the right to make another,hopefully more accurate, prediction right before the actual experiments starts.


8 Success rate is the % of people who wind up taking the pledge within 20 days of the experiment starting.