Summary: Data about where people joining the EA Facebook group first heard of EA.
EA Facebook group moderators Claire Zabel and I, with some help from Julia Wise, have been sending a welcome message to every new member when we add them. By doing so we partly aim to set a good first impression, make people feel welcome, and provide a point of contact for questions.
We also ask them to tell us where they first heard about EA. By doing this, we gather data about where new EAs are formed. Hopefully this data will be helpful for future marketing efforts. As joining the group requires moderator approval, this means we reached substantially every member who joined over the past 31 days. Obviously we're only sampling a subset of EAs - those that join the facebook group - but we are sending the message to ~100% of that subset, and no one else. This compares with the big EA census, which sampled from a much wider group, but it was less clear how representative the sample was.
Between 2015/06/22 and 07/22, 375 people joined the group, bringing us to 6478 members.
Of the ~371 people we messaged*, 216 people responded. This is a 58% response rate. We then tried to fit their response into a broad category like 'Facebook' or 'LessWrong'. Here is the data in chart and table form:
28% Friend
16% Facebook
14% Other
12% Peter Singer
12% LessWrong/CFAR/Eliezer/HPMOR/SSC/MIRI
10% Media Article
4% 80k
4% GiveWell
3% Colleague
3% Philosophy
3% EAG
3% Animal Rights
2% Local Grouo
2% NA
2% Will
1% Akilnathan logeswaran
1% Family
1% Christianity
1% GWWC
We also have a constant problem with spam in the group. A large fraction (Jacy once estimated 80%, but I think more like 30% of the new joiners) of the accounts who attempt to join are fake accounts. In total (since the group began many years ago, and before the current moderators) 403 facebook accounts have been blocked, virtually all of which for spam. This means we must be sceptical of the 6478 members number, as many of these are probably fake accounts using the EA page to make themselves appear more credible. The new members appear to be more legitimate, but we tend to approve new members if uncertain, so there are probably fake accounts among the 375. Presumably these did not respond to our greeting.
Some discrepancies are to be expected, as an artifact of the data collection technique. For example, we had many more people naming ‘Facebook’. The census had LessWrong as the number one source, which probably reflected the prominent link to the survey on LessWrong. However, LessWrong is still a major source in our data, suggesting that the strong result in the census was not just an artifact of disproportionate sampling.
Notably, ‘Peter Singer’ was a pretty major source in our data - much higher than in the census data. Conversely, ‘GWWC’ was a much more major source in the census than in our data. Perhaps indicating the recent bout of attention around EA global, ‘Media Article’ does well in our data, but does not appear in the survey data.
Friendship proves its worth in both data sets.
----
Thanks to Claire to reading a draft of this post. Any errors are, of course, my own.
---
* A very small number of profiles do not allow messages from non-friends
Interesting post. This data seems helpful, but it's probably not worth gathering constantly - maybe on an annual basis or something. Of course, then there are representativeness issues. I would think an online survey would be less effective, but there might be a way to automate this using some software. I don't think Hootsuite can do it but some other app for automating Facebook posts and messages could help.
Thank you for sharing! I think a/b testing this seems like a really good idea. Even just testing the way you are phrasing the question opposed to testing other questions. A static online survey would definitely cut down on the time investment since it will collect all the data for you, however it will definitely cut into your response rate (more clicks = more work).
It seems like continuing to gather this information over the course of all the EA Global meetings and the launch of Will's book would be valuable due to the likelihood of continued rapid growth. Past that it would be more useful to focus on using the data opposed to collecting it.
Right now across both surveys it looks like LW and word of mouth are the best recruiting tools. Continuing to enacting marketing strategies across those two platforms seems like the best course of action. Meaning we should probably be encouraging new members to tell their friends and invite them to meetings. It also seems like a great idea to keep messaging some people to reinforce the welcoming feeling and also because people that have been referred by word of mouth will likely appreciate and need a one-on-one interaction to stay interested or motivated.
A static online survey would definitely cut down on the time investment since it will collect all the data for you, however it will definitely cut into your response rate (more clicks = more work).
It'd be interesting to test that - one factor which will cut the other way is that some people are more comfortable answering an online survey (often selecting preset answer options) rather than getting into a discussion with another human being.
Tying this into the EA census sounds like a good idea as it'd provide a helpful additional subsample, at least for some subset of questions.
Thanks for posting this. I compared this to the data we got from a random sample of the EA Facebook group myself as well. I don't know whether your comparisons above are to our overall results or just the random sample of Facebook we did?
First points re. comparison:
Comments on the actual data: I basically agree that there were some significant differences but overall not enormous divergence.
What to make of the GWWC/CEA wipeout? (For us CEA and LW had basically equal influence, for you the ratio was 2:1 in favour of LW). I would guess the most likely explanation is timing. There was only year between our survey's (results being published), but we were randomly sampling members of the FB group, whereas you were only asking new members. So we'd have sampled a lot of people who're members of the group and have been EAs for a few years, whereas you are sampling solely new people. People involved in EA from close to the beginning will plausibly be much more likely to have heard of it from GWWC. New people, it seems, much less so. (Even if you include EAG and local groups and Will and Tyler personally all in CEA- which would be unreasonable anyway- the numbers don't jump that much). So it seems plausible that CEA is much less of an influence, as a proportion, than it was in the early years. It will be interesting to see if this trend continues and is reflected in our new survey.
Thanks for writing this up! It's very useful to be able to compare this to census data. Did you use the same/similar message for everyone? If so, I'd be interested to see what it was. This sort of thing would also be useful to a/b test to refine it. There is also the option to add people manually, bypassing the need for admin approval; did you contact these people too?
We used:
"Hey, welcome to the Effective Altruism facebook group! If you have a moment, would you mind telling us where you first heard about EA?
Thanks!
Claire
(moderator)"
We are considering a/b testing some new questions, and would love suggestions on different phrasing.
And when someone in the group adds a new member, we still have to approve them. We messaged them as well.
How about using different questions from the EA Census? There was already a big community discussion on what suggestions to include on that, on this forum I think.