It's my great pleasure to announce that, after seven months of hard work and planning fallacy, the EA Survey is finally out.
It's a long document, however, so we've put it together in an external PDF.
In May 2014, a team from .impact and Charity Science released a survey of the effective altruist community. The survey offers data to supplement and clarify those anecdotes, with the aim of better understanding the community and how to promote EA.
In addition it enabled a number of other valuable projects -- initial seeding of EA Profiles, the new EA Donation Registry and the Map of EAs. It also let us put many people in touch with local groups they didn’t know about, and establish presences in over 40 new cities and countries so far.
Summary of Important Findings
The survey was taken by 2,408 people, 1,146 (47.6%) of whom provided enough data to be considered, and 813 of whom considered themselves members of the EA movement (70.9%) and were included for the entire analysis.
The top three sources people in our sample first heard about EA from were LessWrong, friends, or Giving What We Can. LessWrong, GiveWell, and personal contact were cited as the top three reasons people continued to get more involved in EA. (Keep in mind that EAs in our sample might not mean all EAs overall… more on this later.)
66.9% of the EAs in our sample are from the United States, the United Kingdom, and Australia, but we have EAs in many countries. You can see the public location responses visualized on a map!
The Bay Area had the most EAs in our sample, followed by London and then Oxford. New York and Washington DC have surprisingly many EAs and may have flown under the radar.
The EAs in our sample in total donated over $5.23 million in 2013. The median donation size was $450 in 2013 donations.
238 EAs in our sample donated 1% of their income or more, and 84 EAs in our sample give 10% of their income. You can see the past and planned donations that people have chosen to made public on the EA Donation Registry.
The top three charities donated to by EAs in our sample were GiveWell's three picks for 2013 -- AMF, SCI, and GiveDirectly. MIRI was the fourth largest donation target, followed by unrestricted donations to GiveWell.
Poverty was the most popular cause among EAs in our sample, followed by metacharity and then rationality.
33.1% of EAs in our sample are either vegan or vegetarian.
34.1% of EAs in our sample who indicated a career indicated that they were aiming to earn to give.
The Full Document
You can read the rest at the linked PDF! -->
A Note on Methodology
One concern worth putting in the forefront is that we used a convenience sample, trying to sample as many EAs as we can in places we knew where to find them. But we didn't get everyone.
It’s easy to survey, say, all Americans in a reliable way, because we know where Americans live and we know how to send surveys to a random sample of them. Sure, there may be difficulties with subpopulations who are too busy or subpopulations who don’t have landlines (though surveys now call cell phones).
Contrast this with trying to survey effective altruists. It’s hard to know who is an EA without asking them first, but we can’t exactly send surveys to random people all across the world and hope for the best. Instead, we have to do our best to figure out where EAs can be found, and try to get the survey to them.
We did our best, but some groups may have been oversampled (more survey respondents, by percentage, from that group than are actually in the true population of all EAs) or undersampled (not enough people in our sample from that subpopulation to be truly representative). This is a limitation that we can’t fully resolve, though we’ll strive to improve next year. At the bottom of this analysis, we include a methodological appendix that has a detailed discussion of this limitation and why we think our survey results are still useful.
You can find much more than you’d ever want in the methodological appendix at the bottom of the PDF.
In sum, this is probably the most exhaustive study of the effective altruism movement in existence. It certainly exhausted us!
I'm really excited about the results and look forward to how they will be able to inform our movement.
PDF link doesn't exist anymore. @Peter_Hurford
The previous link to the survey results died, so I edited to update the link.
Thanks for running the survey, writing it up, and posting the data. I think this is chiefly valuable for giving people an approximate overview of what we know about the movement, so it's great to have the summary document which does that.
I would have preferred fewer attempts to look for statistical significance, as I'm not sure they ever helped much and think they have led you to at least one misleading conclusion. In particular:
On the contrary, I think the main message from the data is that in the sample collected, they are roughly evenly split. The biggest of the four beats the smallest by less than a factor of two -- this is a relatively small difference when there are no mechanisms I can see which should equalise their size (I would not have been shocked if you'd found an order of magnitude difference between some two of them).
Doing a test here for statisti... (read more)
Thank you for doing this survey and analysis. I regret that the feedback from me was primarily critical, and that this reply will follow in a similar vein. But I don’t believe the data from this survey is interpretable in most cases, and I think that the main value of this work is as a cautionary example.
A biased analogy
Suppose you wanted to survey the population of Christians at Oxford: maybe you wanted to know their demographics, the mix of denominations, their beliefs on ‘hot button’ bioethical topics, and things like that.
Suppose you did it by going around the local churches and asking the priests to spread the word to their congregants. The local catholic church is very excited, and the priest promises to mention at the end of his sermon; you can’t get through to the Anglican vicar, but the secretary promises she’ll mention it in the next newsletter; the evangelical pastor politely declines.
You get the results, and you find that Christians in Oxford are overwhelmingly catholic, that they are primarily White and Hispanic, and tend conservative on most bioethical issues, and are particularly opposed to abortion and many forms of contraception.
Surveys and Sampling
Of course, yo... (read more)
It's worth noting there was also significant domain expertise on the survey team.
Thank you to the survey team for completing what is an easy-to-underestimate volume of work. Thank you also to the many who completed this survey, helping us to both understand different EA communities better and to improve this process of learning about ourselves as a wider group in future years.
I have designed and analysed several consumer surveys professionally as part of my job as a strategy consultant.
There is already a discussion of sample bias so I will leave those issues alone in this post and focus on three simple suggestions to make the process ... (read more)
Thanks for this, and thanks for putting the full data on github. I'll have a sift through it tonight and see how far I get towards processing it all (perhaps I'll decide it's too messy and I'll just be grateful for the results in the report!).
I have one specific comment so far: on page 12 of the PDF you have rationality as the third-highest-ranking cause. This was surprisingly high to me. The table in imdata.csv has it as "Improving rationality or science", which is grouping together two very different things. (I am strongly in favour of improving science, such as with open data, a culture of sharing lab secrets and code, etc.; I'm pretty indifferent to CFAR-style rationality.)
"238 EAs in our sample donated 1% of their income or more, and 84 EAs in our sample give 10% of their income."
I was surprised by this. In particular, 22% (127/588) of people identifying as EAs do not donate. (Of course they may have good reasons for not donating, e.g. if they are employed by an EA charity or if they are currently investing in order to give more in the future). Do we know why so many people identify as EAs but do not presently donate?
I've made a bar chart plotter thing with the survey data: link.
The first 17 entries in imdata.csv have some mixed-up columns, starting (at latest) from
Have you volunteered or worked for any of the following organisations? [Machine Intelligence Research Institute]
until (at least)
Over 2013, which charities did you donate to? [Against Malaria Foundation].
Some of this I can work out (volunteering at "6-10 friends" should obviously be in the friends column), but the blank cells under the AMF donations have me puzzled.
Do you have any sense of the extent to which people who put down 'friend' as how they got into effective altruism might have learned about EA via their friend taking them to a meet-up group, or might have ended up getting properly committed because they made friends with people through a meet-up group? I was just thinking about how I would classify myself as having got into EA through friends, but that you might think it was more accurate to describe it as a meet-up group in Oxford getting me involved.
I'm trying to make sense of all the missing data. It seems very strange to have such a high non response rate (nearly 20%) to simple demographic questions such as gender and student status, and this suggests a problem with the data.
You say here that a 'survey response' was generated each time somebody opened the survey, even if they answered no questions. Does that mean there wasn't a 'complete and submit' step? Was every partially completed survey considered a separate 'person'? If so, was there any way to determine if individuals were opening multiple ... (read more)
Thanks for the survey! An interesting read. One question, two comments:
1 How do I read the graph on p10?
2... (read more)
It's interesting that only around 10% of self-identified EA's report donating 10% or more of their income. That makes me feel less guilty about "only" giving 10%. :)
Is there a way to easily pull out the % of their income that people gave in 2013 in a more granular way - e.g. people who gave 10.0% vs. 10.1% vs 12%? I don't see the data with the currencies converted in github.
This is action relevant for us at Charity Science because assuming GWWC pledgers stick carefully to 10.0% getting them to donate to a particular fundraiser has no counterfactual value.
Tom mentioned that the raw data would be shared (in an anonymized form)?
Also, I recall there being questions about politics - maybe about which other movements we were involved in?