Study results: The most convincing argument for effective donations

I don't have any commentary on this beyond the text, but I love the spirit of this contest! I hope to see many similar studies conducted in the future, inside and outside of the EA community, with a variety of arguments and other media.

Written by Eric Schwitzgebel

The Contest

Last fall, Fiery Cushman and I announced a contest: We would award $1000 ($500 to the author and $500 to the author's choice of charity) to the author of an argument that effectively convinces research participants to donate a surprise bonus payment to charity at rates statistically higher than a control group.

The context was this: Chris McVey, Josh May, and I had several times tried and failed to write arguments that would be effective in increasing participants' donation rates. When we presented participants emotionally moving narratives about children who had been rescued by charitable donations, charitable donations were higher than in a control condition -- but never when we presented ordinary philosophical arguments that donation is good or is your duty. See here for a brief write-up of one version of this paradigm. We wondered whether the failure might just be the result of our inability to write convincing arguments. Therefore, Fiery and I decided to put out the call.

The rules governing entries were somewhat complicated -- see the original contest announcement for details -- but mainly we wanted to see if an argument in favor of donation could be effective without using narrative elements, or mentioning specific individuals, or having vivid emotional content. We couldn't completely forbid emotional content, since even straightforward factual presentation of the facts of human suffering isn't emotionally neutral. But the main idea was just to have ordinary, dry philosophy of the sort ordinarily done by ordinary, dry analytic philosophers.

Our plan was to issue the call, select at most 20 arguments among those submitted, and see if any of those arguments could beat a control condition in which participants read part of a middle school physics text. If more than one argument beat control, the award would go to the author of the argument with the highest mean donation.

After some delay due to the pandemic... we now have a winner!

The winner was a submission cowritten by Peter Singer and Matthew Lindauer, which we will share in its entirety below.

Gathering Submissions and Phase 1 Testing

We were delighted by the community's response to our contest call. We received about 100 submissions, about half from professional philosophers, psychologists, and experimental economists and about half from others who had heard about the contest through social media or otherwise.

We only had the resources to test twenty arguments, so in accordance with our plan, we had to cull the 100 down to 20. In selecting arguments, we considered several factors, including the extent to which the argument was in the spirit of the contest (i.e., a relatively dry philosophical argument) and the extent to which the argument seemed to us well-written and likely to be convincing. We also wanted the arguments to manifest a diverse range of approaches.

So many of the arguments seemed promising that agreeing among ourselves on a balanced set of 20 proved to be a challenge. By the time we had selected our 20 and written and tested the software for administrating the study, the U.S. was shutting down due to the pandemic. We then faced the question of whether we should suspend the study because of the pandemic, out of concerns that responses during the pandemic might not be representative of responses during more ordinary times. We were concerned, for example, that online workers in the U.S. might be facing unusual financial hardship which would lead to lower rates of donation.

We went ahead with the first phase of testing in late April. In this phase, about 2500 participants randomly read one of the 20 selected arguments. After reading the argument, participants clicked to a new page on which they read the following:

Upon completion of this study, 10% of participants will receive an additional $10. You have the option to donate some portion of this $10 to your choice among six well-known, effective charities. If you are one of the recipients of the additional $10, the portion you decide to keep will appear as a bonus credited to your Mechanical Turk worker account, and the portion you decide to donate will be given to the charity you pick from the list below.
Note: You must pass the comprehension question and show no signs of suspicious responding to receive the $10. Receipt of the $10 is NOT conditional, however, on how much you choose to donate if you receive the $10.
If you are one of the recipients of the additional $10, how much of your additional $10 would you like to donate?
[response scale $0 to $10 in $1 increments]
Which charity would you like your chosen donation amount to go to? For more information, or to donate directly, please follow the highlighted links to each charity.

• Against Malaria Foundation: "To provide funding for long-lasting insecticide-treated net (LLIN) distribution (for protection against malaria) in developing countries."
• Doctors Without Borders / Médecins Sans Frontières: "Medical care where it is needed most."
• Give Directly: "Distributing cash to very poor individuals in Kenya and Uganda."
• Global Alliance for Improved Nutrition: "To tackle the human suffering caused by malnutrition around the world."
• Helen Keller International: "Save the sight and lives of the world's most vulnerable and disadvantaged."
• Multiple Myeloma Research Foundation: "We collect, interpret and activate the largest collection of quality information and put it to work for every person with multiple myeloma."

[These charities were listed in randomized order.]

After this question we asked some other questions aimed at exploring the psychological basis of any differences in response. For example, in follow up questions, participants were asked questions about their attitudes and reactions to the text, e.g., how convincing they found the text, whether their attitude changed, and whether they donated more than they otherwise would have.

We also asked some demographic questions, and we asked participants whether they were experiencing unusual financial hardship due to the pandemic and whether concerns about the pandemic had influenced their answers.

Participants who failed a comprehension check (about 4% of participants) were excluded.

In the first round of testing, we had about 120 included participants per argument, across the 20 arguments. The mean donation rate was $2.88 out of $10, which was substantially lower than the mean donation rate of about $3.50 that we have seen in other versions of the experiment. This may have been due to the pandemic: The majority of participants reported at least "slight hardship" due to the pandemic, and 26% reported moderate or significant hardship.

The mean donation by argument varied from $2.22 for the apparently least effective argument to $3.54 for the apparently most effective argument. However, it was not clear whether the arguments actually differed in their effectiveness: A statistical test for difference in means was only marginally significant (ANOVA [19, 2406], F = 1.58, p = .054).

However, our aim in phase 1 was not to reach any definitive conclusions but rather to select the five best performing arguments for further testing. (Preliminary Monte Carlo modeling had suggested that the ultimately best performing argument would likely already be among the top five after 2000 trials.) The best performing five arguments had mean donation rates from $3.10 to $3.54.

Phase 2 Testing: The Winner

In Phase 2, each of the five selected arguments was viewed by about 335 participants, while 471 participants viewed the middle school science text. The results were clear: All five of the arguments substantially outperformed the control condition. Thus, the null results of our earlier research failed to replicate with these new and presumably more effective arguments.

Mean donation ranged from $3.32 to $3.98 for the five arguments, compared to only $2.58 in the control condition. An overall analysis of variance was highly statistically significant (ANOVA [5, 2148], F = 11.8, p < .001). In t-tests at an alpha level of .01 (to correct for multiple comparisons), each argument individually significantly outperformed the control condition (all t > 3.5, all p < .001). However, no difference was statistically detectable among the arguments (in Tukey post-hoc comparisons on the ANOVA).

Here are the results in a bar chart, with error bars representing 95% confidence intervals.

As you can see, the winner in Phase 2 was Argument 9 by a nose. Argument 9 was also the winner by a nose in Phase 1, and thus the winner overall.

Here is the text of Argument 9, which was submitted by Peter Singer and Matthew Lindauer:

Many people in poor countries suffer from a condition called trachoma. Trachoma is the major cause of preventable blindness in the world. Trachoma starts with bacteria that get in the eyes of children, especially children living in hot and dusty conditions where hygiene is poor. If not treated, a child with trachoma bacteria will begin to suffer from blurred vision and will gradually go blind, though this process may take many years. A very cheap treatment is available that cures the condition before blindness develops. As little as $25, donated to an effective agency, can prevent someone going blind later in life.
How much would you pay to prevent your own child becoming blind? Most of us would pay $25,000, $250,000, or even more, if we could afford it. The suffering of children in poor countries must matter more than one-thousandth as much as the suffering of our own child. That’s why it is good to support one of the effective agencies that are preventing blindness from trachoma, and need more donations to reach more people.

When we asked Singer and Lindauer to verify their claim about the cost of treating trachoma, they referred us to Cook et al. 2006, which estimates a cost of $7.14 in 2004 U.S. dollars for a treatment with a 77% cure rate. Singer and Lindauer raised the estimate to $25 to err on the conservative side and account for inflation.

At the end of this post is an appendix containing the other four finalist arguments. We caution against inferences based on specific features of the trachoma argument that are not also shared by these other arguments which performed similarly.

Now although the trachoma argument only won by a nose, in our follow-up questions about participants' attitudes toward the text, it won handily, with a mean attitude of 8.4 on a scale from -21 to +21, compared to means of 3.2 to 6.3 for the other texts and 4.7 for the control text. In other words, participants not only actually donated at rates substantially above the rates in the control condition, but also they said they donated more than they would otherwise have donated and that the text was persuasive. This was not as true for the other texts, none of which were significantly different from control on this measure (ANOVA [5, 2148], F = 16.2, p < .001; in Tukey pairwise comparisons argument 9 beats all others and no other argument beats control).

Conclusion

Hopefully, we can replicate these results after the pandemic is over. In the meantime, I draw the tentative conclusion that the presentation of texts like Singer and Lindauer's can indeed lead people to donate more to charity than they otherwise would have, contrary to what was suggested by some of my earlier null results. Singer and Lindauer's text not only won the contest but stood out in tending to produce positive reactions from its readers, compared to the other arguments we tested.

We will share more data and thoughts later, as well as the texts and results of all tested arguments, but this is enough for today.

Congratulations to Peter and Matt!

Appendix

Argument #3, by Julius Hege (mean donation $3.32):

There are few things that pretty much everyone agrees on. The value of charity is one of those few things. Philosophers are famous for being quarrelsome and agreeing on very little. But in a poll of professional ethicists, 91% responded that a typical person in their position should give to charity. A full 96% report donating themselves last year.[1]

Almost all religious traditions agree as well. For Christians, charity is one the seven virtues. John 3:17 states: “But if anyone has the world's goods and sees his brother in need, yet closes his heart against him, how does God's love abide in him?”

For Muslims, almsgiving (Zakat) is one of the five pillars of Islam. There is also voluntary charity (Sadaqah) going beyond that, which is also widely praised. In Judaism as well there is the concept of Tzedakah, which literally translates to “righteousness”, but often refers to charity. It sees giving not just as an act of benevolence, but as a duty one has to fulfil.

The public also agrees: According to the Charities Aid Foundation, about 88% of people in the UK engaged in at least one charitable action last year.[2] In the US, 86% of respondents believe it’s important for them to continue to give time and money to charity.[3]

Not only is there a wide agreement that charity is good, many people even agree that we should give large amounts. For example, Matthew 19:24 states: “Again I tell you, it is easier for a camel to go through the eye of a needle than for someone who is rich to enter the kingdom of God."

This unanimity is not surprising given the tremendous achievements of international development. Extreme poverty is often defined as living on less than what $1.90 a day can purchase in the developed world. It is often marked by lack of adequate food, drinking water, and basic medicine. In 1980, over 40% of the world population lived in this extreme poverty. Today, only 10% do.[4] In the same time, global life expectancy has increased by more than 10 years.[5] And because this poverty is so extreme, it is also very cheap to fix: An extra $10 for a person in the developed world is nice, but often wouldn’t even pay for a single meal at a restaurant. But it could also buy 5 long-lasting bednets preventing malaria, or deworm 20 children, or stave off malnutrition by distributing iodized salt to 50 people in need.[6]

In conclusion, given how charity is seen as nearly unanimously good and that it can make a larger difference to the world’s poorest, it seems like the case for charity is as strong as it could be.

[1] https://doi.org/10.1080/09515089.2012.727135
[2] https://www.cafonline.org/docs/default-source/about-us-publications/caf-uk-giving-2018-report.pdf,
[3] https://www.philanthropyroundtable.org/almanac/statistics/national-poll
[4] https://ourworldindata.org/extreme-poverty#the-evolution-of-poverty-by-world-regions
[5] https://ourworldindata.org/life-expectancy#life-expectancy-has-improved-globally
[6] https://www.thelifeyoucansave.org/impact-calculator

[Note: Participants saw the footnotes, but the links were not clickable, in accord with the rules of the contest.]

Argument #5, by Adriano Mannino (mean donation $3.84):

Imagine a red button. If you push it, two things will happen. First, you will receive $10. Second, a serious risk of contracting malaria will be inflicted on four children. They might contract the disease, might suffer terribly and might die from it. Would you push the red button?

It seems that pushing this button would be excessively selfish and cruel. By pushing it, you would put your own interest in receiving $10 above the interests of four children in avoiding the malaria risk.

Now, imagine someone randomly puts $10 in front of you. You could take and keep the money, but you’re also offered the opportunity to push a green button for $10 instead. If you push it, four children will be saved from the risk of contracting malaria. Mosquito bed nets will be distributed to them, for a total cost of $10. Sleeping under mosquito nets is a highly effective way to prevent infections in regions where malaria is rife. Two children can sleep under one bed net, and two nets will be distributed for $10. So, instead of keeping the $10, you can use them to save four children from the malaria risk, by pushing the green button. Would you push it?

Failing to push the green button is very similar to pushing the red button. If you push the red button, you receive $10, while four children are exposed to the deadly malaria risk. Similarly, if you fail to push the green button, you receive $10, while four children are exposed to the deadly malaria risk. This trade-off – putting $10 above the interests of four children in avoiding malaria – is precisely why pushing the red button seemed so problematic. Therefore, if you would not push the red button, pushing the green button should be a logical choice. By pushing the green button, you forego $10 but save four children from the deadly malaria risk. This should be a great deal, particularly if you wouldn't push the red button.

Argument #12, by Erik Nook (mean donation $3.86):

One's life is the product of one's choices. Soon you will have a choice to make: Do I take $10 or do I give it to charity? Philosophers have thought of several reasons why donating is the right choice to make today, so I'll tell you about them. But ultimately, the choice is yours. You should feel good about whatever choice you make, but first, take some time to think about why donating might be the better option.

Donating to charity does more "good" than taking money for oneself. Some philosophers think that we should aim to maximize good outcomes in the world, even if sometimes this means that individual people don't get what they would like. This is called utilitarianism. An example of this approach is that it is a good idea to make a medicine that can save 1 million people rather than one that could only save 1 person. Soon you will have the opportunity to give money to a charity that helps a large number of people. These philosophers would say that this should be prioritized over what the $10 could do for yourself. Even though it might be painful to not have $10 in your own life, giving up this money is just the right thing to do "for the greater good".

Selflessness is in itself a "good". Philosophers also think that we should make choices that in themselves are moral. This is the basis of many religious and non-religious codes of ethics. One thing that all religions and codes of ethics agree upon is that giving to other people is a good thing to do. Choosing to give today means that you are making a choice that aligns with what human beings have thought for centuries is a good thing for people to do.

Selflessness can create a culture that encourages other people to do good things. Both psychologists and philosophers have shown that giving is contagious. People who think that other people donate lots of money are also more likely to donate. This creates a culture, a ripple-effect, in which donating leads to more donating. So if you donate today and tell other people about it, you are creating a culture that not only achieves a good thing in your donation but also increases good things happening in the world. You can do a lot of good by donating today.

Selflessness feels good. Lastly, philosophers and psychologists have shown that donating feels good, meaning that you can feel pride, relief, and joy from donating today. Psychologists know that these feelings can improve your well-being and some philosophers would say that these feelings bring meaning to your life and are important to pursue. As such, donating today not only does good things for other people, it also does good things for you.

I hope these ideas get you thinking about the powerful and positive consequences of choosing to donate today. Thank you for your time.

Argument #14, by Jesse Blackburn (mean donation $3.52):

Think about this for a moment: someone you know is suddenly to swap places with one of the 2 billion human beings alive right now who do not have access to clean, uncontaminated drinking water. Or perhaps with one of the 821 million people who suffered from hunger in 2018. What lengths would you go to help the person you know? You might be motivated to stop all you were doing and not rest until you had helped them. Now consider for a moment that you are unable to help. Would you expect others to help? What if someone was able to help, merely by contributing a few dollars, but did not do so. If you think that such a failure is a moral abhorrence, then I ask you to reflect on your own behavior. Would you allow someone else to endure these conditions simply because you do not care to bother yourself with the cost of helping them? I suspect that you have answered no this question, and yet, I put it to you that you do allow this happen. Every single day you have the opportunity to spare a small amount of money to provide a fellow human with the same basic access to food or drinking water – how often have you done this? For most people, our privileged access to clean water and food was not our choice, we were merely fortunate to be born in the right country at the right time, but we can choose to extend that privilege. I am trying to convince you that it is in our power to help and that, if the positions were reversed, if you (or someone you know) needed help and other chose not to help, you would consider them immoral. You are, right now, able to very easily help another human being, consider what you would expect of other people when you make this decision.