Amy, Angelina, & Eli -- helpful data, very clearly presented. Thank you.
I wonder if EA Global also collected any data on age and/or political orientation?
I specifically wonder whether older adults (e.g. over 40) felt as welcome at EA Global as young adults (e.g. under 30), and whether conservatives, centrists, and libertarians felt as welcome as liberals.
This is great stuff. I appreciate you posting some initial results quickly, being careful about what claims you can make right now, signposting what you'll investigate later, and being explicit about what updates you've made.
I'll also echo Lily's comment about dis-aggregating POC. I'd be interested to see POCs breakdown between countries / regions of the world. For example, being a Chinese-American and being a Chinese national are different things.
Interesting, thank you!
Something easy that I would find interesting too: Can you maybe determine the welcomingness among people who are not speakers, don’t volunteer for the conference, and don’t work for EA orgs (I mean, one category of people with no special status boost like that, maybe broken down as above)? I noticed a big difference between EAGs where I was a speaker vs. EAGs where I wasn’t.
Something more difficult: Is there some creative way to get more welcomingness calibration? I was at an EAG once where no one was mean to me or anything, so I ticked the maximum welcomingness. But then I went to a crypto conference (Sustainable Blockchain Summit), and there people actively invited me to the tables where they were chatting, asked me questions, involved me in their conversations, etc. It’s was a wholly different experience. If I had had that experience earlier, I would’ve given that EAG a 3/5 or so instead of a 5/5. I have no good idea how you could figure this out now in the data analysis… Is there maybe an effect that people with more work experience (i.e. maybe more previous conferences) give lower welcomingness scores?
I've felt something similar. I'm roughly thinking of it as being "actively welcoming" as opposed to being "passively welcoming."
Thanks for posting. Appreciate the response and analysis. Would love to see efforts to increase women attendance and participation at EAG and in EA in general. Also curious about the age breakdown of EAG applicants vs attendees.
We use a spreadsheet that tracks racial and gender diversity and representativeness across cause areas. Along the way, we adjust our invites based on who has accepted so far to ensure a breadth of topics and expertise from a range of fields as well as racial and gender diversity across speakers. To be clear, we don't invite people to be speakers solely because of their race, gender, or cause area. We invite people we think will have interesting and useful things to say and have these systems to feature qualified people from underrepresented groups and subject matter experts from various fields.
This feels like a preemptive defense against a strawman of the criticism. I don't think anyone would suggest you're inviting people based solely on their race - that you're just mailing invitations to people selected totally at random among all their co-racialists.
Presumably it is the case that, on the margin, you accepted some people because of their race or gender? That there are some people who were accepted who would have been rejected, if they had been otherwise as good a candidate (aptitude, alignment, interest, etc.) but of another race. And there are some people who were rejected who would have been accepted, if they had been otherwise as good a candidate (aptitude, alignment, interest, etc.) but of another race.
Thanks for sharing this! Helpful in understanding the quality and trends of EAG. More "actionable" dimensions for your consideration and analysis:
Thanks again!
(P.S. I am unable to access the underlying data at the moment of writing this comment, nbd)
Thanks for these suggestions!
(P.S. I am unable to access the underlying data at the moment of writing this comment, nbd)
Thanks for flagging — I've updated the link. Let me know if you have further issues.
CEA has recently conducted a series of analyses to help us better understand how people of different genders and racial backgrounds experienced EA Global events in 2022 (not including EAGx). In response to some requests (like this one), we wanted to share some preliminary findings.[1]
This post is primarily going to report on some summary statistics. We are still investigating pieces of this picture but wanted to get the raw data out fast for others to look at, especially since we suspect this may help shed light on other broad trends within EA.[2]
For EA Globals in 2022, our applicant pool was slightly more diverse in terms of race/ethnicity than our attendee pool (38% of applicants were POC vs. 33% of attendees), and around the same in terms of gender (35% of applicants were female or non-binary vs. 33% of attendees).
For comparison, our attendee pool has about the same composition in terms of gender as the respondents in the 2020 EA Survey and is more diverse in terms of race/ethnicity than that survey.[11]
We had much more racial diversity at EAGx events outside of the US and Europe (e.g. EAGxSingapore, EAGxLatAm, and EAGxIndia, where POC were the majority[12]). Generally, EAGx attendees end up later attending EAGs, so we think the events could result in more attendees from these locations. (However, due to funding constraints and their impact on travel grants, we expect this will not impact EAGs in 2023 as much as it might have otherwise.)
Overall, attendees tend to find EA Global welcoming (4.51/5 with 1–5 as options) and are likely to recommend it to others (9.03/10 with 0–10 as options).
Women and non-binary attendees reported slightly lower average scores on whether EA Global was “a place where [they] felt welcome” (women and non-binary attendees reported an average score of 4.46/5 vs an average of 4.56/5 for men). Otherwise, we found no statistically significant differences in terms of welcomeness and overall recommendation scores across groups in terms of gender and race/ethnicity.[13]
Methodology note: In many cases, attendees did not give us gender or race/ethnicity data, but we calculated the global average across all attendees for the feeling of ‘welcomeness’ and ‘likelihood to recommend’ scores. This is why for instance the ‘overall’ welcomeness for EA Global is lower than the average welcomeness for either POC or non-POC attendees.
We are tracking the lower welcomingness scores experienced by women and non-binary attendees and have shared this with the Community Health team in case it is useful for their gender experiences project.
However, because the absolute differences here are pretty small, and overall ratings remain high, we currently do not think this is a major problem.
Many elements of EAG were designed to encourage welcoming experiences for all attendees. We’ve been holding affinity group meetups for years (e.g. women and non-binary people, people of color, and parents), and those events have often been cited as conference highlights by survey respondents.[14] We generally have at least one community health point of contact onsite, have a code of conduct,[15] have gender-neutral toilets available when possible, and have a parents’ room. Feedback can also be left anonymously in our survey or via the community health form. We take feedback reported to the contact people or in our feedback survey seriously, and where possible investigate cases where attendees feel uncomfortable or unwelcome. As with other demographics, we expect that people who like the events less or feel less welcome at the event might not come back and thus might not be captured in this data.
Overall, speakers and MCs are more diverse in terms of gender than the attendee pool (43% for speakers vs 33% for attendees), and less diverse in terms of race/ethnicity (28% vs 33% for attendees).
We also have data on which speakers and MCs present talks, fireside chats, and workshops[16] (referred to as “Onstage” below). These are the sessions that tend to be recorded and published on our YouTube channel, so will have further effects beyond the conference itself.
Starting in 2018, the Events Team began working with an “advisory board” of people with subject matter expertise across cause areas (currently 36 people). Over time we have consulted this group to varying degrees and also consulted experts outside of this list — including from the team, recent Forum posts, recent grantees, etc.
We start the speaker solicitation process by contacting a few groups: people from underrepresented groups and speakers that we want to approach early for various reasons (like those who are high profile or who we expect to have a lot of competition for their time). We use a spreadsheet that tracks racial and gender diversity and representativeness across cause areas. Along the way, we adjust our invites based on who has accepted so far to ensure a breadth of topics and expertise from a range of fields as well as racial and gender diversity across speakers. To be clear, we don't invite people to be speakers solely because of their race, gender, or cause area. We invite people we think will have interesting and useful things to say and have these systems to feature qualified people from underrepresented groups and subject matter experts from various fields.
Thanks to everyone who has given us input on these processes already. Please help us improve by providing feedback below or anonymously here. Please submit suggestions for content here (feel free to nominate yourself / your organization).
Here is a spreadsheet of the data underlying this post.
There are likely some inconsistencies or mistakes in our coding, but we opted to share while uncertain, rather than delay.
While this post looks only at racial and gender diversity, we recognize there are other forms of diversity e.g. socioeconomic, geographic, age, disability, etc.
Throughout this post, by the term “people of color” we intend to capture people of both non-white and Hispanic backgrounds (i.e. we include white Hispanic attendees as part of this metric). The race/ethnicity field in our application forms used to be a free-text field, so an events team contractor hand categorized this free-text field.
Breaking these figures down further: of registered attendees at 2022 EA Global events 67% self-reported as white, 24% self-reported as Asian, 5% self-reported as Hispanic only, 2% self-reported as Black, and 1% self-reported as multiracial. We note that these figures are more racially diverse than the 2020 EA Survey results, but not reflective of demographics of society generally.
Both the race/ethnicity and gender are fields in our feedback forms.
Speakers don’t consistently give us their gender and race / ethnicity data. When we didn’t have recorded demographic data on speakers, we hand-coded these based on our knowledge of the individual speakers.
It’s worth noting this data could be skewed systematically — for instance, we are unlikely to know if speakers are non-binary but not publicly ‘out’, and we could have mistaken the racial/ethnic backgrounds of some speakers.
For analytics relating to speakers and MCs in this post, if a speaker speaks at two sessions, we’ll count them twice. Unless otherwise specified, we count presenters at our core content sessions (such as talks, workshops, and meetups), but not some less structured or visible session types (such as office hours, guided meditations, private events, or community-initiated events).
For on-stage content at EA Globals in 2022 for speakers only (e.g. not including MCs) 34.5% of speakers were female or non-binary, and 27.3% were people of color.
Application evaluators do not see ethnicity data at the time of application review, but they do see names and country of residence.
We started using tracking spreadsheets in 2018. Unlike the 2022 data in this post, we did not ask Angelina to clean/hand code this data and just pulled what was listed, so some additional inconsistencies might exist. This includes MCs (counted once in general, possibly counted twice in some cases if they host a fireside chat in addition to owning a stage) and speakers (counted each time they speak):
EAG London 2018: 41.5% women/NB, 13% POC
EAG SF 2019: 38% women/NB, 25% POC
EAG London 2019: 51% women/NB, 30% POC
EAG SF 2020: 55% women/NB, 25% POC (ended up as a virtual event due to COVID, not all the speakers participated)
EAG London 2020 and SF 2021: didn't happen, COVID
EAG London 2021: 43% women/NB, 25% POC
2022: covered in this post 43% women/NB, 28% POC
EAG SF 2023: 47.27% women/NB, 29.09% POC
In the 2020 EA Survey, 29% of respondents were women or non-binary, and 22% were people of color (of respondents who provided any race data).
Roughly, EAGxSingapore was ~20% white and ~80% POC, EAGxLatAm was ~25% white/non-Hispanic, ~50% Latin American/Hispanic, 25% other, EAGxIndia was majority Asian (likely South Asian / Indian though many applicants didn’t specify and the category bunches many ethnic groups together). EAGxSingapore speakers were 44.45% women/NB, 53.33% POC; EAGxLatAm speakers were 31.9% women/NB, ~64% POC/Hispanic; EAGxIndia speakers were 31.25% women/NB, 55.55% POC.
For each comparison, we ran a two-tailed t-test assuming homoscedasticity at a 95% confidence interval. It’s worth noting a lot of people don’t note their gender and race / ethnicity in the feedback surveys, and a lack of statistical significance here doesn’t mean that there isn’t a difference in attendee experiences in reality.
Magnify Mentoring got started based on an EAG meetup years ago.
Code of conduct: “At EA Global, EAGx events and social events associated with EA Global or EAGx events, all attendees agree to:
- Respect the boundaries of other participants.
- Look out for one another and try to help if you can.
- Adhere to national and local health and safety regulations, as well as any additional policies we institute for EA Global.
This is a professional learning and networking event. These behaviors don't belong at EA Global or related events:
- Unwanted sexual attention, or sexual harassment of any kind.
- Using the event app to request meetings for romantic or sexual reasons.
- Offensive, disruptive, or discriminatory actions or communication.
We understand that human interaction is complex. If you feel able, please give each other the benefit of explaining behavior you find unwelcome or offensive.
If you’re asked to stop a behavior that’s causing a problem for someone, we expect you to stop immediately. By submitting this form, you confirm that you will adhere to this Code of Conduct, which applies at the conference and all related social events. You can contact us at [email protected] if you have any questions.”
These sessions tend to require some more preparation and often (although not always) are attended by more people.
Thanks so much for doing this analysis and writing this up! I'm curious whether there is a principled reason for using POC as a category, rather than focusing on specific ethnic groups that are underrepresented in EA, especially given what footnote 4 says about the breakdown of EAG attendees who are POC (24% Asian, 5% Hispanic, 2% Black, 1% multiracial). Some people have been critical of the term "POC" because they think it can gloss over this kind of information.
This is also an issue with "Asian" -- it is such a broad category (3/5 of people!) that it combines groups with very different experiences. For example, among Burmese Americans 25% are classified by the US as living in poverty, compared to 6% of Indian Americans.
The question of what "underrepresented in EA" means is also pretty tricky, especially when you're looking at conferences in multiple countries with different groups and histories. This summary seems to handle it by looking at the differences in breakdown between applicants, attendees, moderators, and speakers, but if people are left out of all of these groups that doesn't show up in these stats.
Responded below.
Agreed. Needs work here.
Thanks! I conducted most of the analytics underlying the post. I sympathize with the issue you point out here! The explanation is kind of boring: the data has limitations that make more granular analyses tricky.
In 2022, the EA Global team collected race/ethnicity data exclusively using free-response fields in the application and feedback forms. For this post, we asked assistants working for the events team to hand code each unique response to two fields: (i) whether or not someone is POC, and (ii) which US Census race / ethnicity category this corresponded with. On (ii), I chose this mostly to be consistent with how e.g. the EA Survey in 2020 coded race/ethnicity data, and to allow for easier further analysis.
This second hand categorization is necessarily less accurate than what people would have marked themselves. In particular, our disaggregated race/ethnicity counts are probably less accurate than the “is POC” / “not POC” labeling. As an example, if someone reports they are “Thai / Indian”, I don’t have great guesses for whether they would have marked themselves down as “Asian” or “Multiracial”, but it seems fairly likely to me that they would fit under the “people of color” umbrella. Incidentally, I suspect this kind of issue might be why the EA Survey reports a much larger percentage of multiracial EAs than we do in our attendance numbers.
For speakers, as mentioned in the footnotes most speakers did not give us race / ethnicity data, and so I hand coded a binary “is POC” flag myself. For a variety of reasons coding a more granular flag would have taken much more effort, so we skipped that exercise.
As a second general problem, all of the data we are working with is pretty small, splitting the race/ethnicity data up more granularly makes each cohort smaller, and doing meaningful statistics on small samples is hard.
For the two above reasons, we presented mostly findings on the less granular level here. We might eventually take a look at this question, but I expect this would be a non-trivial lift, so we are currently not prioritizing it over other projects.
As an aside, the events team as a whole is conscious of the dynamic where the term “people of color” hides some important nuance, and doesn’t try to optimize for only this binary categorization when thinking about diversity considerations. (I no longer work on the EA Global team and am passing this on from speaking with the team.)
Reading this makes me think something like "what should the right number be" - I think the implication is like "even if we got to representative numbers of POC, we might still want to focus on specific sub groups" but then it seems like that argument can often be made.
I guess "what are we aiming for here?"
What information would say "this is good?"