Hide table of contents

TL;DR: we (the Forum team) might make a pretty significant change on Frontpage for a month as a test. We’re sharing the outline of the proposal to get feedback from the community. The change we’ll test is hiding “Community” posts from the Frontpage for everybody by default, and moving those posts to a prominent tab. 

Where we are right now: 

We’re probably going to run the test, although we might not depending on the feedback we get as a result of this post and we might significantly change our approach. After the test, it’s pretty unclear to us whether we’ll keep the change — and we might want to extend the test period — but we think that we will probably want to keep something like the change after the month goes by. 

We will review the feedback we get, and if we decide to go for it, we’ll likely start the test within a week or two. 

Summary of the change we’d test: moving community discussion to a separate tab

For the test, we’d move all “Community” posts off the top section of the Frontpage for everyone by default (by setting the default topic filter for “Community” to “hidden,” meaning that people will be able to opt out of the change). We will probably also remove “Community” posts from Recent Discussion on the Frontpage (you will be able to change this back to the default in your user settings). 

Then we’ll probably either add a tab on the Frontpage or a section lower down on the Frontpage, which would feature a limited number of “Community” posts. You could then load more articles or visit the tab. 

There’s a chance that we’ll tweak our classification of what we mean by “Community” posts, but currently, it’s something like what’s described in this footnote.[1] 

Here are two mockups of the layouts we’re considering testing (if we go for one of these, we’ll probably continue testing and improving it):

A “Community” tab

It’s like the normal Frontpage, but “Community” posts won’t show up by default; you can access them via a tab at the top labeled “Community”.

A section for “Community” posts on the Frontpage

​​It’s like the normal Frontpage, with an added section after some number of posts that features around 3 recent and upvoted “Community” posts (sorted like the Frontpage). You’d be able to click “Read more” to view a full list of posts tagged “Community” that you can sort in different ways.

Why consider doing this at all?

Some topics and discussions tend to get a lot more Forum karma and attention than others, as Ben and Lizka outlined in this post. These tend to be: 

  • About the community (because it interests almost everyone at least a little bit)
  • Accessible to everyone, or on topics where everyone has an opinion 

This phenomenon means that posts that interest a smaller fraction of the community — even if they interest their target readers a lot more and are more useful to their audience (and even if you consider that some other posts’ audiences are larger), will get a lot less attention than might be helpful. 

We want to do something to address this imbalance, and we think that separating out “Community” posts might get us a lot of the benefit while still letting important conversations happen. (We checked a lot of posts we thought were getting extra engagement via this effect; almost all were “Community” posts.)

Note that this mechanism “overvalues” “Community” posts whether or not you think that “Community” posts are currently over-valued by the average Forum user (the Forum team has mixed opinions on this[2]); given whatever the community actually cares about, things that are easier to discuss or think about will get more engagement.[3] (See more in this comment.)

We also want the Frontpage to feature different kinds of content that’s particularly interesting to different groups, instead of being full of posts that interest the majority of Forum users to at least some extent (in particular, posts about the EA community). This is how we get cross-pollination between different cause areas or types of work, give people who focus mostly on one cause area the opportunity to engage with excellent content from a different cause area, etc. The status quo makes this significantly harder. (This is not the primary driver of the change.)

We’ve been hearing about the Forum fixating on certain discussions for over a year: this has been making a lot of people sad for a long time.

The Forum team has been hearing about how the Forum often gets inundated with community-oriented posts (and how excellent cause-specific posts get missed) for a long time. (Aaron Gertler, who ran the content side of the Forum until December 2021, tells us that he was hearing this a lot during his time.) This is also often paired with worries that posts that are emotionally charged or on topics where opinions are really strong are particularly likely to encourage people to engage even when they don’t endorse engaging. 

These concerns often look like: 

  • “Hey, I notice that I often get sucked into discussions that are vaguely relevant to me, but which I don’t think are important. I’ve started using the Forum less as a result.” 
  • “When I go on the Frontpage, I just see posts about the community that stick around for a long time. I miss useful posts because they’re a bit more niche and don’t quite neatly fall into the niches I know to look for.”
  • “I wasted hours yesterday reading all the comments on a post to see which comments I agreed with. I wish I hadn’t done that; I wasted time and was upset.”
  • “I don’t want to share the Forum with people who learned about EA and are really excited about working on impactful projects because if they go on the Forum, it’ll look like all we care about is random community stuff. I don’t think it’s all we care about, but I don’t know if new people will understand that.” 
  • Etc. (Note that these quotes are made up.)

We sought out more feedback and got more of the same concerns from people from different parts of the EA community. We also personally agreed with a lot of the concerns that people were sharing. There are also examples of people bringing up similar issues or even very similar solutions (see e.g. this comment) on the Forum and getting a lot of agreement, and Ben and Lizka’s post got a lot of karma (and agreement in the comments — along with some good pushback, but we were not flooded with concerns about the potential changes), which was also a signal that a lot of people were interested in this change.

We’ve tried a number of things to address these concerns: 

  1. Giving people the option to filter topics and customize their Frontpage based on their interests, and promoting this option repeatedly.
    1. Unfortunately, most people tend to leave settings on the default. (Different people on the team have different opinions on the usefulness of this feature.)
  2. In June, we down-weighted “community” posts for new users and logged-out users by default (more). 
    1. We didn’t modify the settings for existing users and now think that this was a mistake. 
  3. In September, we decided to introduce curation.
    1. We think this has been helping a bit, but not really significantly counteracting the effects described. 
  4. We were working on a project to develop subforums in the hope that it would help nurture important conversations relevant to people in different fields. We’ll be posting an update on that project soon, but it doesn’t seem likely to accomplish this goal. 

None of these really solved the issue. 

We’ve had a lot of discussions about this and want to test a change to get some answers.

We’ve had a bunch of conversations about this internally, and a lot of user interviews and conversations with different people from different parts of the EA community. A thing that often happens when you talk to people is that they’ll tell you they’re really excited about a potential feature or change you might push, but then when you make the change, they don’t like it, or don’t use it. We might find out that something like this is going on here. We might also discover that this is a type of change that sounds good in principle, but it turns out that everyone has slightly different interpretations of what it will involve or there are other practical details that are impossible to get right, and almost everyone hates it. 

What do we hope to find out through the test? 

We’d be tracking some things like: 

  • Engagement with different types of posts, and how it might change
  • Whether tricky discussions go better or worse after the change
  • Feedback from the community

If engagement with the Frontpage drops a lot, if tricky conversations are even harder to have well, if we get important negative feedback, or if we see other things that worry us, we will consider the test to be indicating that we should revert the change and go back to the drawing board.

This would all be useful information that’s hard to get without running a test. 

What are our worries about the change we want to test, and what are we doing to mitigate these worries?

Note: for each of these, we’d be very interested in hearing more suggestions about how we can mitigate them. 

  1. If we significantly decrease the visibility of community-oriented discussions, we might miss out on important conversations the community should have (especially things like suggested improvements to our norms, keeping key institutions in EA accountable, etc.). 
    1. To counter this, we will make sure that the “Community” tab is prominent or that “Community” posts get a dedicated spot on the Frontpage, and we’re considering facilitating some conversations like this more directly. We also plan to continue sharing top community posts in the Digest, etc. 
  2. If the “Community” side of the Forum is what is most engaging to people (whether or not they endorse this engagement or this tendency), people may just move towards engaging more with the “Community” tab and engage less with the Frontpage (or engage less with the Forum in general). I.e. it’s possible that a significant part of the particularly useful engagement on the Forum is via a “drive-by” effect where you go on the Forum for a discussion about discussion norms, but accidentally see a technical report on a topic of interest to you. 
    1. We’ll be tracking data on this. 
  3. Separating community-oriented discussion and more object-level discussion may lead to a drop in cross-pollination between the people who focus more on the community and those who focus more on the object-level. E.g. we’ll have posts about event strategy that don’t get useful feedback from domain experts in biosecurity who’ve been to interesting conferences, etc. 
    1. We’ll try to get feedback on this to find out if something like this happened as a result of the change. 
  4. This change might make the Forum more confusing and structurally messy.
    1. We’ll gather feedback on how people understand the tabs/separate spaces, to see if it makes sense to users. Agnes, the UX designer on the team, will be designing the layout; we’re hopeful that it will make sense. But it’ll still probably add complexity, and we’ll need to decide if that’s a price we are willing to pay. 
  5. Newcomers who visit the Forum might be surprised to not see discussions on recent events that are highly relevant to the EA community, or more generally just won’t quickly find discussions they’d expect from a community that’s alive, conscious of its problems, hosting events, etc. 
    1. The Frontpage is pretty active even if we remove “Community” posts, so that’s not a major concern. When there are important events that are getting a lot of coverage and attention even outside of EA, we might often have threads on the Frontpage for discussing them, so that might address some of the concern. We also might come up with better solutions to this problem if we see it realized. 
  6. We may simply land on a significantly subpar solution to the problem (or accidentally solve the symptom of a worse problem that remains unsolved).
    1. If we decide to keep the change, we’ll still be on the lookout for other improvements. 

What sort of feedback is most useful, and how to give it

Please feel free to comment under this post, and we’ve set up two comment polls. You can also reach out privately by emailing forum@centreforeffectivealtruism.org.

  1. ^

    This piece of writing is primarily impactful via the EA community as a phenomenon. It's not significantly relevant to a non-meta organization, field of research, type of real-work-in-the-world, etc. This includes posts about the community, announcements from community-building or infrastructure organizations that are aimed at a familiar-with-EA audience, etc. It doesn’t include things like, “here’s a mindset that helps me really internalize the scope of a problem,” or “A guide on information security as an impactful career path from 80,000 Hours.”

  2. ^

    People on the Forum team have somewhat varying opinions on whether the EA community currently explicitly over-values “meta” posts — posts about the community — too much right now. Several people believe something like: “it would be useful for the average Forum user to spend a bit more time reading posts that are specific to a cause area in EA, over spending a bit more time reading posts about the community-as-a-phenomenon.” We’re worried that this is making us biased here, but we don’t view this as a key motivator of the change.

  3. ^

    We think that a lot of the discussions the community really needs to have are quite difficult, and we want those discussions to get the attention they need. This is either because they’re technical or depend on complicated arguments, or because they’re tricky community issues that we want to discuss rigorously. The karma phenomenon described in the post hurts those (more difficult) discussions in particular, and we want to counteract it.

Comments39
Sorted by Click to highlight new comments since: Today at 11:20 PM
Lizka
1y51
117
16

Agree-vote (✅) with this comment if you are overall optimistic about the test that we’re outlining in the post (e.g. because you think we’ll find out something useful, or because you think that the change will likely be helpful). 

Disagree-vote (❎) if you’re overall pessimistic about the test. 

Please use normal-strength votes (we’ll check for strong-agree-votes and cancel them). (We’re posting two comments as a quick poll to get a sense of what people who might be less comfortable commenting think. Please note that we won’t defer entirely to the results of these polls. See the other comment.)

I think @ChanaMessinger makes a reasonable case that strong votes are much less reasonable in regard to agree/disagreevotes. If you think so here, maybe we shouldn't get them in general. I'm pretty ambivolent.

I've told the LW admins before that I wish there weren't a way to strong-agree or strong-disagree. Or, alternatively, I wish there were hover text that lets you see how many people have strong-agreed vs. weak-agreed vs. weak-disagreed vs. strong-disagreed.

I think ideally I'd also like to see a breakdown into karma buckets, so you can explicitly see how many veteran users vs. new users agreed or disagreed, as opposed to veteran users just quietly getting more voting power.

Maybe something like three buckets: one for people who (based on their karma) probably are pretty new to the forum (or post pretty low-quality stuff), one for veteran users with tons of karma, and one for users who are somewhere in the middle. E.g., an image that shows up when you hover, like (imagining a hypothetical comment with net 12 agreevotes):

 

 

(Obviously all of this would have to be only for future votes, not past ones, since people didn't make their past agreevotes with an expectation that this would be public info.)

I don't have a strong view one way or another on the topic of whether you should be able to use strong-agree-votes anywhere. I don't know what others on the team think. 

I'm quite excited about potentially developing or otherwise adding an easy-to-use poll feature on the Forum, though. It's not an obvious part of any of our major projects for the near future, though, and it's not clear to me that it's worth prioritizing over other improvements (in part because I don't know how much use it would get), so I don't know when exactly we might get to it, if ever — but this is something that would make me personally happy. 

That's interesting, I'd have thought they are more necessary with agree/disagree. Because people with more karma may be more likely to better informed about the topic? But then again people with more karma are also more informed on forum and epistemic norms (upvote/downvote dichotomy?) So maybe both are important

I'm in favor of a weekly cap on how frequently users can use their strongvotes though!

To me it also seems that agree/disagree should be on the left and upvote/downvote on the right. If people are only going to vote on thing, they should prob vote on whether they think the statement is correct, right? 

Lizka
1y42
114
18

Agree-vote (✅) with this comment if you think you will probably be personally happier with the Forum if we make the change that we’re outlining in the post. 

Disagree-vote (❎) if you think you will like the Forum less if we make the change. 

Please use normal-strength votes (we’ll check for strong-agree-votes and cancel them). (We’re posting two comments as a quick poll to get a sense of what people who might be less comfortable commenting think. Please note that we won’t defer entirely to the results of these polls. See the other comment.)

Everyone who agree-voted here, may I ask why you don't configure your own feed to ignore community posts right now?

  1. I want to be able to see them, but not have them be the main thing that I see.
    I guess I could remove them from the frontpage and regularly check https://forum.effectivealtruism.org/topics/community?sortedBy=new but I had never thought about it and feels like effort.
  2. I would want other people to see more of the good object-level posts, and give their feedback on them instead of the community posts. E.g. the current back-and-forth on the evidence on strongminds is to me more interesting and useful than many discussions on polyamory, HBD, or bay area houses. But I'm afraid it doesn't get seen by people that might find it useful, or might have good feedback for the commenters there.
  1. I'm lazy; I am not immune to the phenomenon where users reliably fail to apply optimization to their use of a website, despite their experience improving when such changes are made for them. (I suspect this perspective is underrepresented in the comments because fewer people are willing to admit it and it's probably more common among lurkers.)
  2. I consume content weighted in large part by how many upvotes it has, because that's where the discussion is and it's what people will be talking about. (Also because in my case most of my EA Forum reading comes from karma-gated RSS feeds, though I expect this to be uncommon.) This means that in an equilibrium of most attention going to community posts, I'll read more of them, but I would be happy with a state of affairs that shifted the equilibrium to object-level posts. 

Because I do want to read community posts sometimes and sometimes I might want to read other posts and having two taps that I can choose from depending on which type of post I want to read is a great way to do that.

My need is to have Community posts on the front page in one place, easy to reach and without jumping into the separated Topic section of the forum. This change would suffice my need.

This can also be achieved by clicking on the Community filter and changing the sorting filter from Relevant to Magic (New & Upvoted) but this means one extra click with the sorting, where I cannot change the default sorting.

Clicking on the filter to get into the topics section of the Community brings me into a whole new site without the useful hyperlinks on the left side. Replacing it with a list of who wrote the description of the topic header, which is not useful for me on a daily basis.

I agree-voted with both polls. I recognise the concerns that you outlined with the made-up quotes.

My only real concern is about the definition of "community" posts. To illustrate this, I glanced through some recent posts, selected  a few which I thought were likely to be borderline, and thought that several of them had been tagged as "Community", but didn't have the property of sucking me in an unhealthy way. Examples include 

Native English speaker EAs: could you please speak slower?

“My Model Of EA Burnout” (Logan Strohl)

What's the social/historical story of EA's longtermist turn?

Another post did have that unattractive property (in my view), and was not labelled as community.

If too many "good" posts (whatever that means) are classed as community, I'll just end up looking in the community tab anyway, which might defeat the purpose.

In any case, I'm glad you're giving this a try, and thank you for thinking about this.

Somewhat agree, there are some concerns I have with totally silo'ing community:

1) People might mistag their posts community if they are actually about "building effective altruism". E.g. posts like Community vs Network feel important and useful for professional movement builders (not just local group organizers). I could imagine people accidentally tagging something like this: Evidence from two studies of EA careers advice interventions as a community post or something. 

Perhaps having a moderator occassionally check the community tag could be a good way to ensure that such things don't happen. 

2) There are a number of posts announcing new orgs e.g. Introducing Ayuda Efectiva which would be useful for folks to know to actually help the world directly (e.g. donate in Spain) - and it's also nice to hear about new projects to do good on the frontapage. 

Of course, it's possible that once there is a much more obvious separation folks will know to avoid it. You could also add helpful language when people are drafting a post so they understand what it means for their post to be a community post. (e.g. rather than it just being 1 tag out of 10, you could have it on a separate line to emphasise it more, so people are more aware of it)

Thanks for outlining these concerns. 

Re (1): We currently quickly check tags on posts, and a few people are working on tagging almost all new posts with at least a couple key tags (although there's sometimes a delay — or mistakes/missed tags, which is natural, I think, given that tagging shouldn't be taking up lots of resources). Keeping an eye on the Community section/tab to check that posts there should indeed be "Community" seems like a good idea. I agree that this change would put pressure on the tagging system (it'll be more important to get tagging right). 

Re (2): My hope is that people will check the "Community" section if they want to keep up with "meta" projects, and I tend to share a lot of announcements in the Forum Digest, so that might also be an option for keeping up. Also, just a quick note: announcements from non-"Community"-oriented projects (e.g. a new Charity Entrepreneurship-incubated charity) will still end up on the Frontpage.

I don't think this is about "good" or "bad" posts, it's about whether the post is mainly focused on reviewing/improving the community as a whole or whether it's more about improving individuals or productivity. In that case, "EA burnout" wouldn't be in community, "longtermist turn" would clearly be community, whereas anything about the history of longtermism (e.g. in ancient Rome) would not be.

You brought up a good point that language barrier post being ambiguous.

(I just voted the community tag on that last post -- it wasn't there previously.)

I'm optimistic about this test. Even as a community builder, I find all the community discussion overwhelming. The Forum has become a place I'm less likely to visit because something like half of the discussions don't feel productive for me to engage in.

I'm pro tests.

Thanks for all your work.

This test is a great idea and I hope something like this will get implemented. I'm not a big fan of the tab idea, since community posts will then still be very prominent/accessible. But I do think it's better than what we have today. And in case of the section it would still be great if we could remove that section. Maybe neither a tab or a section is necessary, just show that community is hidden under 'customize feed'. But that might make community posts too hidden.

Thanks for this feedback; we hadn't considered adding an option to remove the section (if we go with that version), and are now considering it. 

(Yeah, we also considered defaulting to hiding Community by default, but I think that would hide "Community" posts too much, and that some people just want to separate the experience of reading object-level posts from the experience of engaging with "Community" posts, and avoid having them compete with each other for attention.)

I'm in favor of running the experiment.

I  would suggest providing people with a week or two notice before implementing this change so that people can get any last community posts out. Otherwise, it might lead to frustration for people who are currently working on posts.

Thanks for this suggestion! I think we might end up moving a bit quicker, as we don't think this change/test will stop people from engaging with "Community" posts altogether — people who want to see "Community" posts will still see them — and because (I think) we're pretty confident that there's a problem here that needs fixing (we're less sure that this is the right solution or that we've got the "problem statement" quite right — we hope the test will give us more info on this front). 

Seems worth trying

At the same time, I don't think the community post / frontpage attention mechanism is the core of what's going on. Which is, in my guess, often best understood as a fight between memeplexes about hearts and minds

I'd personally err towards different subsections rather than different tabs, but glad to see you experimenting to help EA focus on more object level issues!

Hey!

Without disagreeing, I'd like to suggest:

"making the feed less bad in a specific way" is (I expect) only going to be a patch, compared to  "make the feed good".

In different words: If we figure out what we want to optimize the feed for, we won't have to patch each instance of the feed not optimizing for that.

As a metaphor: I think it's a good idea to consider what I DO want in a job instead of looking for a job that doesn't have the 3 features that I didn't enjoy previously.

As an extreme example that I don't actually endorse, I'm writing it to help me point at a more general direction of ideas: Every day, pick 10 users and show them the feed (without any upvotes? without seeing the name of whoever posted it?), and ask them to vote for "this helped me personally" and for "I want more people in the community to read this post". Hopefully "don't show the daily drama to everyone" will be only one of the problems that are implicitly solved by running something like this, many other problems will be wiped out before you even notice they exist, let alone spend months of your time trying to solve each one of them. [reminder: I wouldn't actually do this]

Am I making any sense? Seems like I write more and more lines but I'm not sure if it's any better.

 

Reminder: I'm not disagreeing with you. This seems to be such a big recurring pain point that I do think it's worth attention, and I think it's kind of amazing/insane that we have you to do something about it

I would support trialing option 2, primarily preferring that option due to bullet point 5.

I also prefer option 2.

I agree voted both polls above. That said, I have a concern and some ideas which might mitigate it:

[Agree vote if you like one of the BYPASS IDEAS below, even temporarily ✅

Disagree vote if you think there shouldn't be a filter bypass, even temporarily ❌ ]

CONCERN: After so much discussion I worry that going from all community posts to zero community posts will leave people assuming that recent concerns are not being handled. EA orgs are still charities and I'd still like to hear the actions they take to address things they seem important.

Example: My understanding is that not knowing that actions were being taken for women in the community this is one reason why women felt the need to speak with TIME, because they (validly!) didn't know what actions were being taken in response to all the community discourse.

I have just, the last couple days, been feeling really hopeful that leaders might post more actions they take! A lot of things have been posted recently and I think it's important people see them.

BYPASS IDEAS:

  1. Add a tag for "major community actions" or "actions taken to address community concerns"* or something like that, which is designed to help things end up on front-page still. This subset of posts would differ from say "raising community concerns" or "written responses to community concerns". If the bypass tag is over-abused I could see you cancelling the feature, but maybe it won't be.

  2. If you don't want to give users the ability to self-tag for front-page community stuff, maybe you can make it overt somehow that users can message mods and ask to bypass, if they think their community post may warrant being on front-page still?

Idk if a bypass is necessary forever, but I think it might be useful for the next couple months so as to ease people off the discussions by addressing concerns and making sure people see those concrete actions.

*this was written hastily so I'm sure there is something better here

I think self-tagging would be seriously overused even if the desired categories could be clearly defined. I also think putting the mods in the position of granting/denying special treatment could create controversy. The class of "proposals requiring feedback" in particular is potentially broad, and I dont think we want the mods having to select which proposals to frontpage. I'm potentially more open to this for surveys if they are infrequent.

My guess is that  "describes actions an organization is taking" (as opposed to "actions someone thinks an organization should take") is a relatively objective categorization, and I feel reasonably optimistic about our ability to accurately apply these tags.

That being said, there are a bunch of ways to slice this ontology, and I'm worried we might get a proliferation of exceptions to the rule which will end up making me think that we should stick with the simple "community: yes/no".

Makes sense!

Wouldn't option 2 above mostly resolve these concerns? Community stuff would still be fully visible on the frontpage, just boxed into its own section rather than intermixed.

I do prefer that layout in that sense (not all senses). I'm not sure that's the format the community or staff will end up trying though. They might conclude that section isn't separate enough to solve all the problems with people wasting energy, getting discouraged, and being nerdsniped.

I disagree with the concern: In the proposed solution, we are not going to zero community posts, they are just at a different tab or place at the frontpage but clearly accessible. I would actually be in favour of this change even without any of the concerns regarding 'negativity' from community post, it's a minor change to make the forum better readable!

I think another way of putting this is that I am: -in favor of filtering community posts -wary of burying posts which are officially about "Meta EA: the cause area" or "EA Infrastructure". Like any posts by say, EV's team about new interventions they are trying, I'd probably still want to see.

Maybe the latter wasn't to be included in "community" anyway though

[comment deleted]1y-6
2
1