After many months of hard work from everyone on the .impact team, this year's EA Survey results are finally available.
It's a long document (~25 pages), however, so we've put it together in an external PDF.
Introduction
In November 2015, a team from .impact released a survey of the effective altruist community. The survey offers data to supplement and clarify our perception of people who identify as Effective Altruists, with the aim of better understanding the community and how to promote EA.
The results should be useful to anyone involved in movement-building, analysing the impact of the Effective Altruism community as a whole (especially with reference to donations), or anyone who’s interested in a snapshot of what Effective Altruism looked like in 2015.
Summary of Important Findings
- 2904 sincere people took the survey, and out of those 2352 people would consider themselves EAs. All the following results consider only the people who’d consider themselves EAs. This is three times as many people as last year!
- The most popular way for the people we sampled to hear about EA was Less Wrong (20%), followed by ‘personal connection’ (11%) and ‘Book/article/blog post etc.’ (11%), but 20% of people didn’t answer this question. More people heard about EA for the first time this year than any other year.
- 37% of EAs sampled identified Poverty as the ‘Top Priority’ cause area. The next-most-popular top priority cause was prioritisation, with 9.4% of EAs sampled identifying this as the Top Priority.
- 885 of the EAs sampled donated money to an EA or EA-recommended organisation. The most popular organisations to donate to were AMF, SCI, and Give Directly.
- Total donations (in 2014) from EAs sampled were $6,765,244, with the median being $330; this is very skewed by large donors.
- We recorded 56 donating both last year and this year, and the median increase in donation amount was $296
- 436 (37% of those who answered the question) said yes to ‘Do insecurities about not being “EA enough” sometimes prevent you from taking action or participating more in the EA community?’
- 717 (64% of those who answered) said that EA was welcoming, 103 (9%) said that EA was unwelcoming.
The Full Document
You can read the rest at the linked PDF! -->
A Note on Methodology
One large concern is that we used a convenience sample, trying to sample as many EAs as we can in places we knew where to find them. But we didn't get everyone, and those who replied are not likely to be representative.
This year we initially launched the survey on the EA Facebook page under strict instructions not to share it further, and so we can be fairly sure that the initial group of people sampled were all members of the EA Facebook Group, although not necessarily representative ones. This gives us a benchmark to compare the other subpopulations against.
As we said last year,
"It’s easy to survey, say, all Americans in a reliable way, because we know where Americans live and we know how to send surveys to a random sample of them. Sure, there may be some difficulties with subpopulations who are too busy or subpopulations who don’t have landlines (though surveys now call cell phones).
Contrast this with trying to survey effective altruists. It’s hard to know who is an EA without asking them first, but we can’t exactly send surveys to random people all across the world and hope for the best. Instead, we have to do our best to figure out where EAs can be found, and try to get the survey to them.
We did our best, but some groups may have been oversampled (more survey respondents, by percentage, from that group than are actually in the true population of all EAs) or undersampled (not enough people in our sample from that subpopulation to be truly representative)."
This is a limitation that we can’t fully resolve, although we have tried to make some headway by using the staggered-release mechanism described above.
At the bottom of this report, we include a methodological appendix that has a discussion of the limitations of convenience sampling, and a comparison of the different subpopulations in the survey, ultimately concluding that the data we have doesn't allow us to detect a statistically significant difference between different subpopulations in donations and primary cause choice, although certain demographic indicators - such as meat consumption and gender - are different between the subpopulations.
-
Overall, this is the most comprehensive survey yet of people who identify as Effective Altruists, and should help to inform discussion about the movement for the next year.
The original results are hosted on a site that no longer works, so the results have been moved here: https://rethinkpriorities.org/s/EASurvey2015.pdf
Hi Peter, I'm getting an "Oops! That page can’t be found." in this new link too. Would love to be able to read the full report!
Excellent job, thanks for taking the time to analyze the data so well. Kudos!
I'm glad the survey asked questions about how welcomed people feel. It's really important to help people feel welcomed in order to motivate them to engage with EA well. It's concerned to me that 36% percent felt that the EA community was not actively welcoming, and of those a quarter felt that the EA community was actively unwelcoming.
This number is very similar to the outcomes of another question that makes me highly concerned, "Do insecurities about not being "EA enough" sometimes prevent you from taking action or participating more in the EA community?" The fact that 37% answered yes make this an obvious area of concern, as the survey report correctly identified. I'd be highly interested in seeing what is the correlation between these two questions, and would be willing to bet money the correlation is not insignificant.
Of course, the likely number for these 2 questions is much higher, since many more of the people who feel this way would not take the survey. I think Julia's recent post has some great ideas about how to address this issue.
I'd also like to see the correlation between those people who feel this way and the amount and proportion of income of donations. Like others, I'm concerned that the median number is only $330. Are the people who perceive a high obligation to give and insecurities about not being "EA enough" also the ones not donating much or are they donating more than the average EA? That would impact how to address the question of how to get the median donation up.
On the positive side, I see that the average proportion donated by EAs is 7.5% of their annual income, which is 350% higher than the average donated by Americans. Nice to see that number.
Overall, I think that being more welcoming would cause more people to be more positively engaged with the EA community, identify more strongly as EAs, and be more eager to give their money and time. This should be a win-win!
For the analysis of donations, which asked about donations in 2014, I'd like to see the numbers for people who became EAs in 2013 or earlier (including the breakdowns for non-students and for donations as % of income for those with income of $10,000 or more).
37% of respondents first got involved with EA in 2015, so their 2014 donations do not tell us much about the donation behavior of EAs. Another 24% first got involved with EA in 2014, and it's unclear how much their 2014 donations tell us given that they only began to be involved in EA midyear.
That is a very good point, and ties in to vipulnaik's point below about starting the survey collection time just after the start of a year so that donation information can be recorded for the immediately preceding year.
I've quickly run the numbers and the median donation in 2014 for the 467 people who got involved in 2013 or earlier was $1,500, so significantly higher than that for EAs overall. This is not including people who didn't say what year they got involved, so probably cuts a few people out who did get involved before 2014 but can't remember. Also if we have constant attrition from the EA movement then you'd expect the pre-2014 EAs to be more committed as a whole
This is a very good point and is making me lean towards vipulnaik's suggestion for future surveys, as this problem will be just as pressing if the movement continues to grow at the rate it has done.
Thanks for looking this up quickly, and good point about the selection effect due to attrition.
I do think that it would be informative to see the numbers when also limited to nonstudents (or to people above a certain income, or to people above a certain age). I wouldn't expect to see much donated from young low- (or no-) income students.
Very interesting. I think there's a number of reasons why the $1,500 figure might be biased downwards or updwards as an estimate for median EA donations/charitable savings after exposure to the movement. It might be too low if many people are just saving to give rather than consuming (attempting to measure this may be useful in the future). It might be biased upwards if those who joined the movement earlier are predisposed to give more than the average EA or those who take the survey are more likely to donate than the average EA.
Regardless, I think the evidence suggests there are a lot of untapped donations among current EAs. I think trying to increase current EA donations should be a priority.
I noticed a contrast between the framing of a couple different parts of the survey results:
I think it would be more reasonable to say that the median donation is a "significant problem", and the gender ratio is suboptimal but I'm not terribly concerned about it. I believe we as a movement should generally be less concerned about how many of us are men and more concerned about how much people are donating.
Certainly many people have good reason for only donating a little, but on average, I'm sure we can donate more than $333, and we should encourage ourselves to do better.
I suspect that the gender ratio is largely because most EAs are currently science or comp sci. One problem with raw gender ratio as a metric is the more successful we are among male dominated demographics, the worse this number will look. Trying to be more welcoming is worthwhile, but if we really want to increase the number of women then EA needs to make inroads in female dominated demographics. I believe that this deserves much more attention. I also have some concerns that the gender ratio might be indicative of EA getting locked into particular demographics, which could greatly limit the potential of the movement as a whole.
Following on, one better metric would be the gender ratio among EA recruits of a particular demographic. If 60℅ of comp sci people were male, but 70% of comp sci EAs were male, then that would be more indicative of an issue then if both figures were 80%.
Actually, the most concerning post of the survey is how few deontologists we have. Deontologists are the majority of the population, so we need to get some deontologists in order to obtain mass appeal.
I agree. $333 seems pretty lame, I wonder what the median donation is of an average person in the same income/age spread and how far off of our numbers it really is. It's a fairly important question to ask. If the main good EA does is helping people pick better charities rather than increasing total donation amount that might suggest a change in outreach strategy.
I agree with the caveat that the $333 figure is much less worrisome if it's due to a high number of student or people working for nonprofits.
I have two questions.
(1) The number of donors per charity was included and even broken down by referral type, but the total amount of money moved per charity was not (I believe) disclosed. I'm wondering if this is data you intentionally left out of the analysis (for privacy reasons) or just happened not to include. I would be interested in that data since it can help convey the seriousness of donations. If privacy considerations are an issue, you might wish to disclose the number only for the charities that got at least ten donors.
(2) Somewhat related to Dan Keys' point: the feedback time between the year of donations (2014) and the time the results are released (July 2016) is larger than ideal. You mentioned that the reason for having to do it this way was that the survey was conducted prior to the end of 2015. I'm wondering if it might make more sense to conduct the survey around the end of January, and then have the results released a little before the Giving Season of the next year. Is this something you've considered, and/or are there other ways you expect the feedback loop to be shorter in subsequent years?
(1) We didn't ask people how much money they donated to individual charities, that's right. The data is available in the github repository for the project - search for 'github' in the report
(2) I agree that conducting the survey at the start of the calendar year would be better. Whether we would do that depends to some extent on whether we'd want to wait six months until we start the next survey. We are tightening up the feedback loop - we're improving the code used to analyse it every time. This year the survey was handed around quite a few people - we hope next year to have a dedicated person who can focus entirely full-time on it.
Thanks! On point (1), I downloaded the raw data shortly after posting the comment and realized that the existence of people who have donated to multiple charities makes the analysis difficult. Apologies for not doing that before posting the comment.
If you waited until Jan 2017, would you ask about both 2015 and 2016 donations?
Here are a few ideas for improvement that may be worth considering:
This Year's Survey Results Document
1) Add page numbers for ease of reference
2) Revise the document so that the Key Findings on page 2/3, the table on page 11/12, the table on page 12/13, the table on page 18/19, the table on page 28/29, and the abbreviations on page 31/32 each display on a single page
3) Add percentages to the age histogram on page 14, the location table on page 15, the career table on page 18/19, and the graph on page 22
4) For the table on page 26, add a "Total" column and a "Total" row that show both the number and the percentage of respondents choosing each option for each of the two questions
Future Surveys
1) Ask people for the dollar amount they donated to each broad cause area to get an estimate of what percent of money donated by EAs goes to each cause area
2) Ask people who changed their career or career plan because of EA which career type (earning to give, direct work, research) they are now pursuing instead
3) It might be interesting to ask people whether they tell non-EAs about EA and if yes, the number of non-EAs they have convinced to become an EA
4) It might be interesting to have a marital status question as well as a question about whether those who are married are married to another EA
Overall it's a great survey with very useful information. Thank you for all of your effort!
Thank you for the feedback!
It may also be a good idea to a) change "meta" to "movement building" because the latter term is clearer and b) change "prioritization" to "prioritization (causes, careers, and charities)" to clarify that it is more than just cause prioritization.
The low median donation by EAs suggests that more needs to be done to raise this level. We now have a large list of EAs (the EA facebook page) with ~11k EAs. I think it would be worthwhile to conduct some message testing experiments on the list if possible. We might send messages/articles promoting a greater level of donations to a random subset of the list then later survey the facebook group to ask how much they have donated during the time period since the messagea. Then we could compare the average treatment effects of the various messages used.
At the Effective Animal Advocacy Symposium, Garrett Broad pointed out in his talk that the 2015 Survey of Effective Altruists did not ask about race, which is worrying given how overwhelmingly white the movement is. To my knowledge this makes at least two public critiques of the movement on this specific topic.
He points out that the best way to deal with race issues is not to ignore the issue, but to bring it front and center. Could we please be sure to include a question about race on the 2016 version of this survey?
EDIT: Here's an image. I'll upload a video of his talk once ACE puts the videos of the conference online.
EDIT: The video is here. It's titled "Advocacy for Education" and Garret Broad's section of the talk begins at 33:20.
As an update, this year I did not need to contract any virtual assistants to convert currencies, because our data was standardized enough that it could be done computationally. To convert all the currencies, I used the open source CurrencyR R Package which I also created.
Hi, I notice the content section of the pdf do not quite match up to the actual subheadings in the document. In particular the contents list a section on 'What benefits do people get out of being in a local group?' which is not in the doc. Just wanted to check that nothing has been missed out of the write-up.
Otherwise - AWESOME job guys - keep up the good work - and feel free to ask if you need funding to make this happen for future years.
Great work. I won't repeat the praises others have already given though, and go straight to my comments :-)
One question i'd like to see more explicitly addressed in such surveys is, How counterfactual has the EA movement actually been?
I've only skimmed through the report, but it didn't seem like you do much to address this question. I'm much less concerned about the $333 median donation than i am about the $6.8 million total donations. That's about 20% of what GiveWell attributed directly to their research in 2014 [http://blog.givewell.org/2015/04/13/givewells-money-moved-and-web-traffic-in-2014/] (still only about half of it, if you discount Good Ventures grants).
Your table showing that GiveWell is how the largest number of EA respondents got more involved with effective altruism suggests that there might be a lot more people out there hearing about GiveWell and whatnot from other means who are essentially effective altruists, but don't know about it, or simply don't identify with the movement for some reason. Do the responses from people who have heard of effective altruism but don't identify themselves as effective altruists shed any light on why this is so?
I'm wondering whether one could run the same survey in parallel to, say, GiveWell followers and the donor bases of some EA-vetted charities, but with the questions, 'would you consider yourself an EA' replaced by, 'what factors did you take into account in choosing the charity you donate to' --- with the term 'effective altruism' tabooed, if you will.
Of course this only touches on the 'individual donations to charity' dimension of effective altruism, and i'm not sure what fraction of the total impact of the movement that should represent --- or even how to measure it.
I hope this is helpful :-)
Great job! My suggestions would be to include questions covering: a) career path and career changes, b) ways how individuals first got involved and are currently involved (e.g. via an EA hub event, if they attend any, or get involved in the online EA community) and c) what events/resources they use and would like more of/would get them more involved. Hopefully addressing the identity/discomfort issue!
Great work! For next year, maybe include a question about time spent? This could even be disaggregated into the different ways of being effective, including volunteering for an effective organization, influencing people, contributing to the movement in general (fora, etc), enabling an effective altruist, etc. I guess if you want to solicit different definitions of EA, you could ask the people why they think they are EA.
Will the data (and analysis code, if applicable) be publicly released?