Hi Ben,

Thanks for the comment!

We will definitely keep refining the categories year-by-year depending on which seem more or less significant.

“The Sam Harris and Joe Rogan podcasts were done by Will as part of the promotion campaign for DGB while he was working at CEA/80k so could arguably be coded as DGB/CEA/80k”

I entirely agree that Will’s podcasts can be counted as CEA/80K/DGB/Will in terms of assigning credit.

I’m not sure what that tells us about how we ought to design the survey though. In terms of coding open comment responses: as I noted, how to classify people’s open comment answers is somewhat fuzzy, many comments ambiguously fit multiple different options and those numbers can’t be directly compared to the fixed category responses. Your podcast case is a perfect example since, as you note, if someone writes in “Joe Rogan Podcast” it could be classified as ‘DGB’, ‘CEA’, ‘80K’, ‘Podcast’ or ‘Will’ (or depending on our interests, we could code it as ‘mass media outreach’ or ‘online’ and so on). I could go back through the open comments and try to code things as CEA or 80K related (per your specification), but it will be a pretty vague and heterogeneous category with a lot of edge cases like these, that will make it hard to interpret. Explicit references to existing EA orgs or Will etc. were coded as such.

Also, crucially, a lot of the comments weren’t specific enough to code in that way. In the specific case of Sam Harris podcasts, for example, at least one person mentioned the Peter Singer episode (no-one explicitly mentioned Eliezer’s AI episode that I saw, but in principle some people might have first heard through that one), which means technically we can’t code every reference to Sam Harris’ podcast as Will’s- though I think it’s entirely reasonably of you to assume that many of the non-specific comments were referring to Will’s. In terms of refining the categories: note that DGB and 80K were already available as options and respondents chose to select “Other” and write in a response anyway. It would be difficult to change the fixed categories, in such a way that people who would otherwise write “Joe Rogan podcast” would instead select “CEA”/“80K” etc. as a category. For one thing, people need to know (and remember) that, when they hear Will on a podcast, this should be counted as 80K or CEA or as part of the marketing for DGB or whichever category. They may also just feel that ‘Other: Podcast’ better captures their case than DGB or 80K. I think it would be difficult to specify all the things that could reasonably be counted as CEA’s work in a given fixed option (e.g. ‘80,000 Hours, inc. Doing Good Better, 80K Podcasts, Will MacAskill podcasts etc.’), but we’ll continue reviewing the results and trying to think of the best options.

I agree that in light of this year’s results, including Podcast as its own category next year may well make sense. We could also split out Book/Article etc. into multiple options. Though doing this might actually give us less information, in terms of attributing responses to CEA/80K/Will etc., as if people just select ‘Podcast’ as a fixed response, rather than writing in an ‘Other’ response, we won’t know which Podcast it was. We could include/require an open comment to specify further in addition to fixed responses, but requiring open comment would be significantly more onerous for respondents and might reduce response rate, so it’s a tricky balance. Alternatively we could include more fixed options (Podcast: Will; Podcast: 80K/Rob Wiblin; Podcast: Other, and so on), and ditto for other specific books, websites and so on (so far, Doing Good Better is the only one we’ve extended this treatment to), but of course that might make the question too unwieldy.

“Presumably some of the books / articles / talks are also other materials produced by the organisations or press coverage they sought out - does that seem right”

I definitely agree this is right in terms of assigning credit: no doubt EA orgs such as CEA should claim a lot of credit for vicariously bringing about lots of other outcomes (e.g. lots of Personal Contacts are presumably influenced by the prior work of EA orgs), but again, I don’t think there’s a way to capture that in people’s First Heard responses. And again, when open responses explicitly referred to an EA org, then it was coded accordingly, as well as coding it as ‘Book’ or ‘Article.’.

“Likewise, maybe 'search' and 'facebook' should be removed as categories, because they're channels you use to find the other content listed. Presumably everyone who found out about EA through 'facebook' likely saw a post by a friend, so should be a personal referral, or saw a post by one of the orgs, so should be coded as an org.”

I think it’s pretty plausible that they should be removed next year (given the relatively small numbers attached to them). Although it doesn’t necessarily follow that the answers could neatly be split off into “personal contact” or “an EA org” because people may remember that they saw something on Facebook, but not which org was responsible (a lot of comments were pretty vague like this): so getting rid of “search” and “facebook” would probably mean a lot more “Other” responses.

“I'm also surprised to see https://www.effectivealtruism.org/ isn't listed”

We’d certainly be open to including this website if you are particularly interested in it (although then there’s a question of which other specific EA websites should or shouldn’t be included ). For what it’s worth we didn’t receive a single “Other” response referring to it, that I saw, but maybe people classified this as “Search” or something else.

Thanks Ben. Yeh, this is 3 people in 2009 and 3 people in 2010 (out of 2473 responses to these questions overall). There are a handful of similar errors for Doing Good Better. Every year, there are a few people who seem to get the years wrong in this way (alongside a lot of responses saying explicitly that they don't remember).

Anecdotally, (both in the survey and elsewhere) I find a surprising number of people confuse CEA, 80K and GWWC (not to mention, Rethink Charity, its various projects and Charity Science).

Thanks Ben, this seems like a great suggestion. We're going to be talking about different levels of engagement a lot in the subsequent series of the post (with regards to cause preference in particular), but will make sure to put together an analysis on this specifically.

Hi David,

We briefly looked into this and there was basically no interesting relation between “getting into” EA via the different factors. This isn’t particularly surprising to me, since a priori, for most of these factors, I wouldn’t expect getting more involved in EA via one factor to make you more likely to get involved in EA via a different factor relative to the other factors. For example, if you get more involved in EA via 80,000 Hours, should this make you more likely to get involved with local groups or online EA than if you got involved via GWWC? Or, if you get more involved in EA via 80,000 Hours, should this make you more likely to get more involved in EA via local groups than get more involved via online EA? For the most part, I’d expect not, and that was pretty much what the data shows. Most forms of getting more involved are just weakly positively correlated with most other forms of getting more involved, which is what I’d expect (if you get more involved in EA in any one way you are just slightly more likely to report getting involved in any other way).

The only variables between which there seemed to be any substantial connection (and even these were weak) was that a local group being important for one getting more involved was associated with a personal contact being important for one getting more involved. This makes sense, and is arguably almost trivially true, since it seems like being involved in a local group necessarily entails having personal contacts. In the cluster analysis you could arguably_vaguely_ make out a ‘nothing much selected’ cluster, a ‘local group and personal contact (but also a modest amount of online, 80K, GWWC, and book, blog etc.)’ cluster and an ‘everything else inc. especially online EA, 80K, GWWC, and book, blog etc.)’ cluster, but as noted, there wasn’t much definition within the clusters.

Conversely, a case where I think that cluster analysis does make sense and where the results were more informative was looking at categories of group membership as we do here.

I think another instance where looking at the relationships between reporting getting more involved in EA in different ways and other outcomes is the series of Multiple Correspondence Analyses that we included in the Cause Selection post. If you examine these, then you can see, for example, that there is a correspondence between “getting more involved” via online EA, a personal contact or local group and lower prioritisation of climate change (and the converse pattern for AI/x-risk).

With regards to the question of looking at more engaged subgroups, we looked at the relationship between first hearing about EA in different ways and levels of involvement previously. You’ll note that we found little evidence that different routes into EA lead to substantially higher or lower proportions of people becoming highly engaged EAs. Rather, the number of highly engaged EAs seemed pretty closely proportional to the total number of EAs recruited from each source.

When looking at the multi-select question, we do find some significant differences, though as in the case of ‘first heard’ data, we would expect this to be confounded, most importantly by how long respondents have been in EA. Still, it seems like significantly more people who report getting more involved via a book or blog, etc., EA Global, local EA group, online EA or personal contact are members of the EA Forum than not (across all time periods). You can also compared the raw percentages of people involved in different ways across Forum members and non-Forum members. I don’t think this suggests anything in particular about any causal relations between being involved in one way and being involved in another though.