I strongly recommend using the impact-adjusted plan change metric rather than the unadjusted one for 80,000 Hours. Those figures:
Sep 2014 to Aug 2015 - 184.8
Sep 2015 to Aug 2016 - 631.3
Sep 2016 to Aug 2017 - 1202
There's also our newsletter growth. New subscribers each year:
2014 - 262
2015 - 23,000
2016 - 76,000
2017 so far - 57,000.
Ok, done. Thanks. Impact-adjusted numbers are fairer, since that is what you are actually targeting, though there is some subjectivity in the impact adjusting process.
Couldn't you put in both types of data so readers can draw their own conclusions?
Thanks. It's not so much that it's a matter of what we're targetting - I think some of these plan change are more than 100x more valuable than others. If you take the raw number of customers a company has, when some customers spend 100x more than others, the total number of customers could look good while the business is actually shrinking. Our raw plan change figures are dominated by the smallest, least important plan changes so could be quite misleading in future.
Hi, I was thinking that the sum of local EA group facebook numbers might serve as a good proxy for the size of the EA movement per se.
I am part of the local team building an EA presence in Brussels, and for local groups this is a useful metric to measure success in one sense i.e. as an easy-to-generate quantification of the size of our group at any one time (as well as the historical trend) as well as for the movement as a whole.
It could also be combined with other metrics e.g.
with impact (e.g. GiveWell donations, 80k Hours career changes) to assess communications effectiveness.
with time (of the local EA team) and cost invested to achieve to assess operational effectiveness.
If compared across local groups, the metrics would highlight local success stories and potentially where groups might need help.
Regards,
Allister.
the sum of local EA group facebook numbers might serve as a good proxy for the size of the EA movement per se
This is an interesting proposition, but one thing that will limit its usefulness, I think, is that lots of EAs are members of multiple local group Facebook groups, presumably either to show support or out of interest in their content. Aside from that, many members of the online groups appear to be not engaged in the local EA community (and perhaps not really engaged in EA at all): for example, EA London has around 2000 members of the Facebook group but many fewer people who are actively engaged and attend events and so.
It could also be combined with other metrics e.g. with impact (e.g. GiveWell donations, 80k Hours career changes) to assess communications effectiveness. with time (of the local EA team) and cost invested to achieve to assess operational effectiveness. If compared across local groups, the metrics would highlight local success stories and potentially where groups might need help.
The Local Groups Survey did this to some extent: measuring (self-reported) number of group members, Pledges, Career changes, funds raised or donations influenced, among other things. We don't publicly release a breakdown of particular groups, of course, but we did look at the correlations between different variables and performance on different metrics. As you'd expect there was a fairly good correlation between success on different metrics, though with plenty of exceptions. I agree it would be valuable to have more systematic investigation of group performance of this kind to identify trends and where things seem to be working particularly well or not well.
Hi, In case helpful for considering the additional Facebook information, I have a bunch of data on EA social media presence to help me compare growth in London to other locations, including a lot of downloaded Sociograph data from 2016.
For example the EA Facebook group size over the last year:
03/06/2016 _ 10263
13/01/2017 _ 12070
10/06/2017 _ 12,953
Obviously you'd expect these things to grow as people join then do not leave (but might ignore it), even if the movement was shrinking.
The effective altruism subreddit is growing in traffic: https://i.imgur.com/3BSLlgC.png (August figures are 2.5k and 9.5k)
The EA Wikipedia page is not changing much in pageviews: https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=all-access&agent=user&start=2015-07&end=2017-08&pages=Effective_altruism
The subreddit stats used to be public (or rather, moderators could choose to make them public) but that option was removed by Reddit a few months ago.
https://www.reddit.com/r/ModSupport/comments/6atvgi/upcoming_changes_view_counts_users_here_now_and/
I discussed Reddit stats a little bit in this article: https://www.wikihow.com/Understand-Your-Website-Traffic-Variation-with-Time
Thanks. Good idea to include the subreddit. How do you get those stats? We can add traffic from the r/smartgiving subreddit too (r/effectivealtruism precursor) to go back prior to 2016.
I have EA Wikipedia pageview data in my post already. :)
You can find the stats by going to the right of the page in moderation tools and clicking "traffic stats". They only go back a year though. Redditmetrics.com should show you subscriber counts from before that, but not activity.
Thanks, I added the Reddit stats to the article!
Vaguely interesting data point: In January 2018 I listed ~240 people in London who I considered - based on personal acquaintance - to be members of the EA community ("EAs"). Compare that with only 66 Londoners taking the EA Survey 2017.
Lots of caveats that mean you shouldn't assume the current global EA community has ~6,680 (=240/66*1837) members e.g. although I'm now a group leader, I imagine there are dozens/hundreds of EAs in London who I don't know; I expect the London community is growing faster than others because it's a popular destination for young, talented, ambitious people; I don't know what proportion of survey takers identified as EAs; I think EA London promoted the survey more than any other group did etc.
Still. Vaguely interesting data point.
In mid-March 2017, EA Wikipedia page views hit 3-4 times as many views as any day in preceeding or succeeding months...What happened in March?
I have been using PredictionBook for recording predictions related to GiveWell money moved; see http://effective-altruism.com/ea/xn/givewell_money_moved_in_2015_a_review_of_my/#predictions-for-2016 for links to the predictions. Unfortunately searching on PredictionBook itself does not turn up all the predictions because they use Google, which does not index all pages or at least doesn't surface them in search results.
Note to self for next year: ACE tracks money moved.
These data are helpful, but they somewhat miss what we really care about.
First, we care more about growth in impact on top causes than the numbers of people involved etc.
I think the key story here is that Open Phil has ramped up donations from $30m to over $120m (4-fold growth in a year), and is expected to increase that several times more in the next few years, but this would be easy to miss in the presentation above.
I'd also prefer to try to look at even more fundamental measures of progress in the top causes, though this is hard since they'll be qualitative. For instance, I think taking AI risk mainstream as a field 2015-2017 has been a huge success.
When it comes to measures of the number of people engaging in EA, we also have to be careful. Many in the community have stepped back from increasing the number of people involved & media outreach, towards increasing the quality of engagement, as reflected in our recent survey: https://80000hours.org/2017/11/talent-gaps-survey-2017/
See some defence of this idea here: https://www.centreforeffectivealtruism.org/blog/the-fidelity-model-of-spreading-ideas/
This means some of these declines in web hits and so on might actually be intentional. We could have easily driven up visits to the EA Forum or wikipedia page if we had tried.
That said, unfortunately, we don't yet have a good measure of the quality of engagement, so it's hard to know if that strategy is working either (though I feel like we're making good progress within 80k at least).
You should really add non-'global health and development' OpenPhil grants as that's now almost as much as all the other donations here listed together (e.g. $105m since the start of 2017, vs $60m through all of 2016 and little before that). Easy to get all the data here: http://www.openphilanthropy.org/giving/grants .
Adding in that the growth rate looks much faster.
Good idea. I added that in.
This post was co-authored by Peter Hurford and Joey Savoie.
The EA Survey team at Rethink Charity (including myself) recently released initial data from the 2017 EA Survey and will have more to follow it up. KBog made a comment on the EA Forum noticing that the 2015 EA Survey had 2532 participants, whereas the 2017 EA Survey only had 1837 participants. Does this mean that EA is declining in growth?
It’s hard to say, and the EA Survey team will have more analysis on this, looking deeper at what the data in the EA Survey tells us about EA membership growth and churn, if anything. However, Joey Savoie (unrelated to the EA Survey team) and I (Peter Hurford) were curious to look a bit more at other metrics of potential growth to get a clearer picture. Prior analysis on this has been done by Vipul Naik at the end of 2016 and by Eric Yu at the beginning of 2016 (in a post that I sponsored the creation of).
Joey and I thought it would be a good time to try to re-check the data and see if things have changed. We compiled the following data from various sources to look at EA growth rates.
|
Type of data |
Sep 2014 to Aug 2015 |
Sep 2015 to Aug 2016 |
Sep 2016 to Aug 2017 |
|
The EA Facebook group (via sociograph.io)[1] |
1547 posts 471 authors 920 commenters 1924 reactors[2] |
1374 posts 521 authors 1055 commenters 2710 reactors |
696 posts[3] 400 authors 1089 commenters 2872 reactors |
|
618 |
770 |
1150 |
|
|
Number of 80,000 Hours impact-adjusted significant career changes |
184.8 |
631.3 |
1202 |
|
Google interest in “effective altruism” (relative scoring)[4] |
16.94 |
28.4 |
38.2 |
|
EA wikipedia data (desktop pageviews)[5] |
51,100 |
70,500 |
77,800 |
|
New EA Newsletter sign-ups |
Didn’t really exist[6] |
11,167 |
35,554 |
|
EA Forum pageviews |
309,961 |
345,202 |
390,463 |
|
New EA Reddit subscribers[7,8] |
308 |
869 |
1484 |
|
Type of data |
January 2014 to Dec 2014 |
January 2015 to December 2015 |
January 2016 to December 2016 |
|
Non-OpenPhil GiveWell donations |
$30-40M (math) |
||
|
OpenPhil GiveWell donations[9] |
$70M |
||
| $30.2M
|
$34.2M |
$126.4M |
|
|
OpenPhil + non-OpenPhil GiveWell donations |
$21.3M |
||
|
9044 |
14,287 |
No data available[10] |
|
|
Total recorded money actually donated (not pledges) from Giving What We Can members |
$7.0M[11] |
No data available |
|
|
80,000 Hours Newsletter Subscribers |
262 |
23,000 |
76,000 |
|
Type of data |
EA Survey 2014 |
EA Survey 2015 |
EA Survey 2017 |
|
813 |
2532 |
1837 |
|
|
Total amount of recorded donations in EA Survey |
$5.2M |
$6.8M |
$9.9M[12] |
Additional Facebook Data
As an admin of the EA Facebook group, I was also able to access the built-in statistics, but they only go back 60 days. Right now, as of 30 August 2017, the EA FB group has about 8629 “active users” in the last 60 days as defined by Facebook (user has viewed, posted, commented, or reacted in the group). The last 60 days also saw 99 posts, 2073 comments, and 7042 reactions. 339 members were added in the past 60 days, to a total of 13,407 members right now. Comparing the past sixty days to the past 28 days shows roughly linear growth, but I would not expect to be able to see a trend in only two months of data, even if EA was growing very quickly.
This data is not very useful right now since it only goes back 60 days, but it might be valuable as a time capsule, since a year from now we could re-do this post and compare the numbers for the group as of 2018 to these archived numbers.
Commentary
Overall, it’s hard to get a good sense of a trend from only two to three data points for each trend. It looks like EA is growing somewhat and that the growth rate is somewhat linear, but it varies depending on the source and I would definitely want to see more data to be sure. A lot of the growth rate can vary depending on how much marginal resources organizations like the Center of Effective Altruism put into intentionally growing a particular source (e.g., the EA Newsletter) versus letting a source grow organically (e.g., the EA Forum).
I don’t really know what kind of growth rate I was expecting prior to seeing this data, so I can’t say if this does or doesn’t support my hypotheses about movement growth. Joey and I are trying to hold off from having firm additional opinions until we can look at the data more. I think it would be ideal to create concrete predictions around where these numbers will be within one year or so (e.g., see here, here, and here). We certainly invite discussion and predictions on this.
Acknowledgements
This post was co-authored by Peter Hurford and Joey Savoie. Thanks to Vipul Naik and Issa Rice for putting together some of the source data used in this report and referring us to the correct places for getting EA Forum and EA Wikipedia pageview data, and thanks Pascal Zimmer for supplying data on the EA Newsletter. Thanks to Kerry Vaughan for providing data on pageviews for the EA Forum in 2017, thanks to Rob Wiblin for some 80,000 Hours statistics, and thanks to Zeke Sherman for pointing out Reddit statistics.
Endnotes
[1]: Sociograph does offer to show member growth over time as a paid feature, which I got via a 14-day free trial, but the data looked incorrect and unusable.
[2]: I believe “reactors” refer to the number of people who like, haha, love, etc. I am not sure if this includes shares, but I do not think that it does.
[3]: It’s worth noting that the EA Facebook group engages in reasonably heavy moderation of Facebook posts and probably (my intuition as a Facebook mod) rejects about five posts for every one post that reaches the page. Given that moderation may have changed over time, it’s not clear how much to read into this decline in the post count.
[4]: These numbers are not search volumes -- they’re the mean relative “score” for that year, relative to the search volume for the highest day between January 2004 and the end of August 2017.
[5]: See some more wiki data here and here.
[6]: The EA Newsletter was relaunched and first had regular content and marketing in October 2015.
[7]: Both r/EffectiveAltruism and r/smartgiving have been simultaneous EA subreddits since September 2012. r/smartgiving was the default EA subreddit until an intentional migration on 28 Feb 2016. From Sep 2016 - Aug 2017, r/effectivealtruism had +1484 subscribers (1068->2552) and r/smartgiving had +93 (1466->1559); from Sep 2015 - Aug 2016, +869 (199->1068) for r/effectivealtruism and +230 (1236->1466) for r/smartgiving; and from Sep 2014 - Aug 2015, +91 (108->199) for r/effectivealtruism and +308 (928->1236) for r/smartgiving. I will use r/smartgiving numbers for the 2014-2015 period and r/effectivealtruism numbers for all periods after that, to reflect the transition. Note that this growth will therefore involve some inherent double-counting as people who were subscribed on r/smartgiving re-subscribe on r/effectivealtruism. Pageviews for reddit were calculated via http://redditmetrics.com/.
[8]: Reddit admin stats go back a year for pageviews. Looking at this, there were ~65,350 pageviews between Sep 2016 and August 2017. We can keep this number in this comment as a time capsule and then if/when we do a Growth Metrics 2018 we can compare the pageviews to this number.
[9]: Money from the Open Philanthropy Project is counted for the year in which the grant is announced, which may be different from the year the grant is decided or the year the grant money is actually dispersed.
[10]: GiveWell has not yet published their Metrics Report for 2016 data.
[11]: These numbers come from different sources, so I don’t know if they are directly comparable.
[12]: The EA Survey 2017 recorded donations from both 2015 and 2016. This is 2016 data only. There was $6.7M in donations recorded for 2015 in the 2017 EA Survey.
Regarding the rate of EA Facebook group posts - a year or two ago there was an effort to introduce moderation to slow the rate of low quality posts (often from entirely new people) that added noise and reflected poorly on the group. The lower engagement there seems to be a sign of success and professionalisation rather than slow movement growth.
Definitely. I note that in endnote #3, but it definitely should be pointed out. The heavier moderation started in December 2015.
Sorry I missed the endnote. I think in that case, where the change is most naturally interpreted as neutral or positive, it's worth giving it a stronger caveat than a footnote, for people just skimming the numbers. :)