[ Question ]

What metrics may be useful to measure the health of the EA community?

by nonzerosum 22d14th Nov 20191 min read4 comments

15


Does this already happen? If not, should it? And with which metrics? If it already happens, what metrics are used to assess the health and changes in health of the EA community?

New Answer
Ask Related Question
New Comment
Write here. Select text for formatting options.
We support LaTeX: Cmd-4 for inline, Cmd-M for block-level (Ctrl on Windows).
You can switch between rich text and markdown in your user settings.

2 Answers

Interesting question.

I think there are essentially two different angles here: how good is the EA community at achieving its stated purpose, and how healthy are the members.

For the first one, how many people are donating at least 10% of their labour income is an obvious test. The extent to which EA research breaks new ground, vs going round in circles, would be another.

For the second presumably many standard measures of social dysfunction would be relevant - e.g. depression, crime, drug addiction, or unemployment. Conversely, we would also care about positive indicators, like professional success, having children, good family relationships, etc. However, you would presumably want to think about selection effects (does EA attract healthy people) vs treatment effects (does EA make people healthy). If we (hypothetically) made some people so depressed they rapidly drop out, our depression stats could look good, despite this being clearly bad!

Another issue is judging whether someone is a member of the community. A survey could be unrepresentative if it doesn't reach enough people - or if it reaches only peripherally attached people.

Some metrics for local groups we gathered for this post were:

  • How welcoming do people find the group?
  • Do those focused on a non-prioritised cause feel welcome?
  • Do people from relevant (context-dependent) minority groups feel welcome?
  • Do group members feel like they are part of a community?
  • Do group members feel supported by the local group?
  • Have there been any major community issues?
  • How were they dealt with?
  • How did this affect other group members’ perception of the group?

Ideally, you could aggregate these across local groups and get a sense of local group health.

You would also probably want to look at more "EA achieving its system goals" kind of metrics - i.e. Lark's suggestion of donating 10%, but also career changes and volunteer projects taken on (i.e. organizing a local group, volunteering on an EA-aligned project).

Looking at how the two metrics correlate would be fairly important (for example if it turns out an EA hub haslower community health but higher output, this would be an important finding)