The 'community' tag is problematic

Arepo

34

Recent example - an op ed piece on the AI safety pipeline having too many researchers is labelled community, a post advocating for more AI field building by an OP grant maker is not.

Lizka

10mo

21

Replying quickly, speaking only for myself^[1]:

I agree that the boundary between Community & non-Community posts is (and has always been) fuzzy
1. You can see some guidance I wrote in early 2023: Community vs. other^[2] (a "test-yourself" quiz is also linked in the doc)
  1. Note: I am no longer on the Online Team/an active moderator and don't actually know what the official guidance is today.^[3]
2. I also agree that this fuzziness is not a trivial issue, & it continues to bug me
  1. when I go to the Frontpage, I relatively frequently see posts that I think should be "Community" that aren't, or vice versa
However, the Community tag/distinction/section was not "a way to separate out FTX-scandal-related posts", IMO
1. Fwiw, I was (also?) really worried about this change when I was working on it, both because I was concerned it would suppress useful criticism/reflection/etc., and because I wanted to make sure my decision-making wasn't somehow secretly tainted by various biases.
  1. I'm writing quickly here — not taking time to dig up my old docs or trying to really remember what was going on in early 2023^[4], but I vaguely remember e.g. calling @Ben_West🔸 at some point when I was feeling particularly stressed, to double check that we endorsed how we were approaching the change
  2. See also footnote 7 here, and footnote 2 here
2. More:
  1. A lot of the thinking/earlier attempts had happened before the FTX collapse
    1. "We’ve been hearing about the Forum fixating on certain discussions for over a year [as of Feb 2023]: this has been making a lot of people sad for a long time. [...]"
      1. [from here]
    2. And we'd already down-weighted "community" posts for new & logged-out users (before the FTX collapse); IMO this was always a pretty hacky feature that had a bunch of flaws
    3. The community tag itself had existed for a while before that
  2. And, quoting from myself here (from July 2023):
    1. Just a quick clarification: I don't think this was a "change to lower community engagement." Adding the community section was a change, and it did (probably^[1]) lower engagement with community posts, but that wasn't the actual point (which is a distinction I think is worth making, although maybe some would say it's the same). In my view, the point was to avoid suffocating other discussions and to make the Forum feel less on-edge/anxiety-inducing (which we were hearing was at least some people's experience). In case it helps, this outlines our (or at least my) thinking about it.
3. (i.e. +1 to @Lorenzo Buonanno🔸 in the comments)
Quick takes on where things are today:
1. I would like to see more "Community" posts / discussions
2. I think the Community section overall is probably still (quite) positive (but tagging could probably be improved)
  1. Off the top of the head, the two key things I'd probably look at /think about, if I were considering removing it (or doing something else), are (1) whether it seems like it's really tanking engagement on Community posts, or preventing them from being written (I'd maybe chat to likely/potential authors about this, look at the numbers, etc.), and (2) how active users (i.e. readers) feel about the Community section^[5]

^{^}
I.e. I'm not speaking for the Online/Mod teams here, and didn't run this comment by anyone.
^{^}
(I vaguely remember making and linking a public version of this doc somewhere at some point, but couldn't quickly find that.)
^{^}
In fact it looks like I can no longer add or remove the Community tag from posts. I'm still in the Slack; a few people sometimes flag questions about edge cases there.

^{^}

(...a lot!)

^{^}

I'd guess that most people still prefer it. The change got a lot of support, both before and after (and this is rare).

Julia_Wise🔸

10mo

4

I agree that separating out community posts was not just a reaction to the FTX situation. Early in CEA's time running the Forum, the community section was an entirely different page, as you can see in this 2019 Wayback capture.

Lorenzo Buonanno🔸

10mo*

15

The community tag was originally introduced as a way to separate out FTX-scandal related tags.

I don't think that's true, based on what CEA staff were posting publicly and some conversations I had at the time.

Some relevant posts and comment threads:
1. https://forum.effectivealtruism.org/posts/wvBfYnNeRvfEXvezP/moving-community-discussion-to-a-separate-tab-a-test-we#Why_consider_doing_this_at_all_

2. https://forum.effectivealtruism.org/posts/dDudLPHv7AgPLrzef/karma-overrates-some-topics-resulting-issues-and-potential

3. https://forum.effectivealtruism.org/posts/2jYDXwqSj87ZjLtwy/follow-and-filter-topics-and-an-update-to-the-community#3__The__Community__tag_and_topic

4. https://forum.effectivealtruism.org/posts/irhgjSgvocfrwnzRz/should-the-forum-be-structured-such-that-the-drama-of-the#GNLKxKvcijjcxSiRG

When I was a moderator, my understanding was that the community tag was more about separating posts related to EA as in "doing good" from posts related to EA as in "a specific community of people". E.g. People uninterested in the community but still interested in AI Safety would still be the target audience of a post on "AI safety talent development"

That said, there were plenty of ambiguous cases, and users can tag any of their own posts as community when posting, so I agree that it's somewhat inconsistently applied.

Arepo

10mo*

2

These were all (with the exception of 3) written in the six months following FTX, when it was still being actively discussed, and at least three other EA-relateed controversies had come out in succession. So I probably misremembered FTX specifically (thinking again, IIRC there was an FTX tag that was the original 'autodeprioritised' tag, whose logic maybe got reverted when the community separation logic was introduced). But I think it's fair to say that these were a de facto controversy-deprioritisation strategy (as the fourth post suggests) rather than a measured strategy that could be expected to stay equally relevant in calmer times.

NickLaing

10mo*

10

I agree its problematic, but I think it is better to have it rather than not. Personally I like the separation as it helps my brain to organise the topics, and I feel like community posts still get plenty of attention. You may well be right that CEA is biased (its hard not to be) and the criteria could be
made clearer. I'm also not sure community posts get less attention (forum team can tell me). Some might healthy Karmas on current community posts!

I did poorly on your test, but those are the hard cases you chose. Generally in life its not best to make sweeping decisions based on rare/hard cases.

Fran

10mo

6

+1. I don't know about attention, but I do think the 'community' tag has a vibe of being 'less important' than posts without the tag. I think this is mostly a feature of the community itself and what users want the forum to be primarily focussed on. I don't mind this, even though I personally enjoy community posts just as much, and I also like the separation. But, if my vibes-based sense is correct, then that does make the system by which posts are tagged slightly more consequential. So I think it's good that Arepo looked into this and is bringing it up. Thanks for doing that!

Arepo

10mo*

4

You may well be right that CEA is biased (its hard not to be) and the criteria could be
made clearer.

My suggestion if the current separation is kept would be to reallow community tagging of posts, but require it to go above a certain threshold and/or have a delay, so that posts don't bounce back and forward between the two feeds.

I'm also not sure community posts get less attention (forum team can tell me).

FWIW I suspect both that being tagged community causes reduced attention, but that they get less attention overall since many low-karma posts slide off the feed without having time to get tagged. I.e. getting attention causes a post to be (more likely to be) tagged community.

Jason

10mo

2

If the separation is going to continue, I'd prefer it be entrusted to (elected? appointed but independent-of-CEA?) stewards. My concern is that community tagging might end up being voting by a different name (users will be less likely to tag things they like).

Owen Cotton-Barratt

10mo

8

If you want independent criteria-based judgements, it might realistically be a good option to have the judgements made by an LLM -- with the benefit of having the classification instantly (as a bonus you could publish the prompt used, so the judgements would be easier for people to audit).

Will Aldred

10mo

4

Fyi, the Forum team has experimented with LLMs for tagging posts (and for automating some other tasks, like reviewing new users), but so far none have been accurate enough to rely on. Nonetheless, I appreciate your comment, since we weren’t really tracking the transparency/auditing upside of using LLMs.

Owen Cotton-Barratt

10mo

6

That makes sense!

(I'm curious how much you've invested in giving them detailed prompts about what information to assess in applying particular tags, or even more structured workflows, vs just taking smart models and seeing if they can one-shot it; but I don't really need to know any of this.)

Larks

10mo

6

As a general principle I think the forum team should reject ~ all requests for additional bureaucracy.

NickLaing

10mo

4

I don't think i agree with this general principle. I think there are few serious requests for extra beurocracy here on the forum and they can probably be assessed one by one on merit?

If the requests were overwhelming then maybe I'd agree

Bit I'm not a libertarian ;)

Dylan Richardson

10mo*

8

I'm very pro-deprioritizing of community posts. They invariably get way more engagement then other topics and I don't think this is only an FTX related phenomenon. Community posts are the manifestation of in/out group tensions and come with all of the associated poor judgement and decorum. The EA forum's politics and religion.

Obviously they are needed to an extent, but it is entirely reasonable to give the less contentious contributions a boost.

Arepo

10mo*

10

I feel like people are treating the counterfactual as 'no way to filter out community posts'; whereas the forum software currently allows you to filter for any given tag, and could easily be tweaked to (or possibly already allows you to) filter out a particular tag.

So the primary counterfactual isn't 'no separation' it's 'greater transparency and/or community involvement in what gets tagged "community"'.

OllieRodriguez

10mo

6

It looks the post in question is now tagged 'Community'.