Effects of major events on EA activity


21


ericyu3

Peter proposed using time series of EA activity (Google searches, GWWC pledges, etc.) to see whether major events like EA Global and GWWC’s pledge drive are associated with spikes of interest in EA. I looked at the data, and found many of the correlations that you would expect (GWWC does get more pledges during their pledge drive), but also a few surprises (e.g. EA Global has few obvious effects).

Although there wasn’t enough data to get a particularly detailed picture of EA growth, many spikes in activity coincided with major events, suggesting that the events caused the increases. To see how different events corresponded to spikes in activity (or didn’t), see the annotated plots below.

Data sources

  • Facebook posts weighted by comments, likes, and shares (from sociograph.io)

  • GiveWell website traffic and mailing list sign-ups (from Tyler Heishman at GiveWell)

  • GWWC pledges (from https://www.givingwhatwecan.org/dashboard)

  • Google search interest in “effective altruism” (from Google Trends)

Analysis

Dates of major EA events:


  • Effective Altruism Summit 2014: 8/2-8/3, 2014

  • GWWC pledge drive 2014–2015: a few weeks around 1/1/2015

  • GWWC pledge drive 2015–2016: a few weeks around 1/1/2016

  • EA Global 2015: 7/31-8/2, 2015 (SF), 8/14-8/16 (Melbourne), 8/28-8/30 (Oxford)

  • Doing Good Better published: 7/28/2015 (around the same time as EA Global)

  • The Most Good You Can Do published: 3/15/2015 (published on Amazon 4/7/2015)

  • Peter Singer’s TED talk: March 2013

  • GiveWell top charities announced (2014–2015): 12/1/2014

  • GiveWell top charities announced (2015–2016): 12/18/2015

 

For each metric, I collected data on cumulative activity over time, filled in missing data points using linear interpolation, downsampled into 1-week bins, calculated week-over-week changes in cumulative activity, and normalized the weekly rates to the activity’s average rate over all weeks. I plotted the following time series:


  • Facebook posts

  • Total number of times the Facebook posts were shared

    • Includes shares that occur after the week ends

  • New GWWC members

  • New GiveWell mailing list members

  • Google search interest

Plots for the most informative metrics can be viewed with Tableau—click on the images below for links. All metrics are normalized to their mean value over time; for example, a value of 1.2 for Facebook posts means there were ~20% more posts that week compared to the average week in the whole period since February 2014.

Facebook activity (orange = posts, brown = posts weighted by shares):


 

GiveWell mailing list signups (red) and GWWC pledges (green):

 

 

Google searches for “effective altruism”:

 

Some metrics were left out of the plots, either due to incomplete data (GiveWell website traffic) or lack of informativeness (the rate of Facebook likes is highly correlated with the rate of posts).


GWWC and GiveWell have major spikes in pledges and mailing list signups during the holiday season. However, GWWC did not have a spike in the 2013–2014 season, when they did not have a drive—this strongly suggests that their pledge drives are causing the spikes in pledge rates. GiveWell’s spikes occur earlier, probably because they announce their top charities before GWWC’s pledge drives start. Facebook and Google search activity don’t spike at the same time, and surprisingly, GWWC’s monthly traffic (not plotted) isn’t significantly higher during their pledge drives.


Other interesting trends:


  • GWWC gains members at a fairly constant rate outside of pledge drives

  • The number of Facebook shares is more informative than the number of posts, since the rate of posting varies very little

  • EA Global and Doing Good Better don’t have as large of an effect as you might expect. There seems to be a spike in highly-shared Facebook posts around that time, but not in the overall number of posts.

  • Google search interest seems to depend mainly on media coverage. EA Global/Doing Good Better had a large effect, but the 2014 Effective Altruism Summit did not. Unfortunately, since EA Global occurred at the same time as the book’s publication, it’s not clear which event caused the spike.

  • Google search interest rose quickly until mid-2015, but is fairly stable now and has stayed fairly constant since (although only a limited amount of time has passed). Three of the largest spikes occurred when:

    • Peter Singer gave his TED talk

    • EA Global was held and Doing Good Better was published (The Most Good You Can Do didn’t have an obvious effect)

    • GiveWell published their updated list of top charities for 2015 (no similar spike happened in 2014)

  • Facebook posts and shares grew quickly between 2014 and mid-2015, but are growing much more slowly now

Total activity

To give an idea of how significant the spikes are, here are the approximate average weekly rates of each activity:


Google Trends interest since 2013

~40% of the December 2015 peak

FB group posts

30.7

FB group likes

306.2

FB group comments

186.1

FB group shares

15.8

GWWC pledges

5.0

GiveWell traffic

8160.4

GiveWell mailing list sign-ups

11.8

Conclusions

Overall trends

The single most surprising pattern was that different types of activity seem to be driven by very specific things. There’s no event that causes a spike in all metrics, and only EA Global and GiveWell’s 2015 top charities announcement are associated with a spike in more than one.


Another surprising trend is that none of the activity metrics have grown much since mid-2015. This could mean that EA’s growth is slowing down. More optimistically, this could simply be because we don’t have enough data yet to detect for growth. For GWWC membership, arguably the most important metric, it’s still too early to tell whether people have been joining at a higher rate this year.

“Awareness” vs. “membership”

Google Trends seems to be the best indicator of how much publicity effective altruism is getting and how many people are aware of EA, since the level of interest spikes strongly when books, talks, and news articles are published.


On the other hand, the total number of GWWC pledges and GiveWell mailing list subscriptions seems like a better measure of “how many effective altruists there are”. (The most concerted effort to find as many self-identified EAs as possible is .impact’s annual EA Survey, but this only lets us see the impact of events months after the fact, and it’s difficult for an annual survey to isolate the effects of individual events throughout the year.)


However, this doesn’t necessarily mean that having more events to boost GWWC/GiveWell affiliation would increase the number of EAs. For instance, people who pledged during GWWC’s pledge drives might have been planning to join anyway at a different time—see Linchuan Zhang’s post for a more detailed discussion of this issue.


The rate of GWWC pledges outside of pledge drives and the level of Google search interest grew at roughly the same rate, both doubling from 2014 to 2015. If this trend continues, it would suggest that growth is reasonably balanced between (1) public visibility and (2) attracting people who agree with EA enough to donate 10% of their income.

What other kind of data would be the most useful?

The three main types of metrics—Facebook posts, organization membership, and interest on Google Trends—gave distinct and complementary information about EA’s growth, so having another type of metric would likely be more informative than additional web traffic or mailing list signup statistics. Ideally, we would have detailed information on when, where, and how much people are donating, but this data will be much harder to collect.