Please see the disclaimer at the bottom of the post.
Posts that interest everyone — or discussions where everyone has an opinion — tend to get a lot more Forum karma (and attention) than niche posts.
These posts tend to be
- about the EA community
- accessible to everyone, or on topics where everyone has an opinion
Why does this happen?
There are different groups with different niche interests, but an overlapping interest in the EA community:
When a post about the EA community is published, many people might have opinions, and many people feel that they can vote on the post. Most people upvote, so more people voting usually means that a post will get higher karma.
Similarly, if the topic of the post is something that doesn’t require particular expertise to have an opinion about, lots of people feel like they can weigh in. You can think of these as “lowest-common-denominator posts.” This is related to bike-shedding.
This leads to some issues
- This misleads people about what the Forum — and the EA community — cares about
- 10 of the 10 highest karma posts from 2022 were community posts, even though less than ⅓ of total karma went to community posts.
- When someone is trying to evaluate the quality of the Forum, they often go to the list of top posts and evaluate those. This seems like a very reasonable thing to do, but it's actually giving a very skewed picture of what happens on the Forum.
- Because discussions about the community seem to be so highly valued by Forum readers, people might accidentally start to value community-oriented topics more themselves, and drift away from real-world issues
- Imagine an author posting about some issue with RCTs that’s relevant to their work — they’ll get a bit of engagement, some appreciation, and maybe some questions. Then they write a quick post about the font on the Forum — suddenly everyone has an opinion and they get loads of karma. Unconsciously, they might view this as an indicator that the community values the second post more than the first. If this happens repeatedly or they see this happening, they might shift towards that view themselves if they defer even a bit to the community’s view.
- Now imagine this happening on the scale of the thousands of people who use the Forum; these small updates add up.
- This directs even more attention to community-oriented, low-barrier topics, and away from niche topics and topics that are more complex, which might be more valuable to discuss
- Karma is used for sorting the Frontpage: higher-rated posts stay on the Frontpage for longer. This is useful, as it tends to hide the most irrelevant posts, and generally boosts higher quality content — more people see the better posts.
- But because posts that hit the middle sections in the Venn diagrams above get more karma, they tend to stick around for longer, which then gets them more karma, etc.
(We didn't try to make this list of issues as exhaustive as possible.)
Note that karma is not perfect even within a much more specific topic — pretty random factors can affect a Forum post’s karma, and readers aren’t always great at voting, but that is a separate issue. (We might write a post about it later.)
Solutions we’re considering or exploring
- Create something like a subforum or separate tab for “community opinion” posts, and filter them out from the Frontpage by default
- Or otherwise move in this direction
- Rename “Top” sorting to more clearly indicate what karma actually measures
- We tend to have a somewhat higher bar for sharing “community” posts in places like the Digest, largely for these reasons
Note: We (Lizka and Ben) think most of our coworkers on the CEA Online Team more-or-less agree with the post, but there are a variety of opinions. We’re currently at 90%+ that we will do something to address this phenomenon, but at much lower confidence about what specific thing we will do.