Our plans for hosting an EA wiki on the Forum

by Aaron Gertler4 min read2nd Mar 202159 comments

123

WikisEA Forum (Meta)Get Involved
Frontpage

We are in the process of implementing a major project on the Forum — turning our current system of tags into a full-fledged “EA wiki”.

Under this system, many of the tags used for posts will also serve as articles in the wiki. Many articles in the Wiki will also serve as tags that can be applied to articles.

However, there are exceptions in both directions. Some tags don’t make sense as wiki articles (for example, “EA London Update”). And some articles are too narrow to be useful tags (for example, Abhijit Banerjee). These will be marked as “wiki only” — they can be found with the Forum’s search engine, but can’t be used to tag posts.

The project is made possible by the work of Pablo Stafforini, who received an EA Infrastructure Fund grant to create an initial set of articles.

Why is an EA wiki useful?

EA content mostly takes the form of essays, videos, research papers, and other long-form content. If you want to find a simple definition of a term, or a brief summary of a cause area, you often have to find a particular blog post from 2012 or ask someone in the community.

A wiki can serve as a collection of definitions and brief explanations that help people efficiently develop their understanding of EA’s ideas and community.

It can also host more detailed articles that wouldn’t have a reason to exist elsewhere. Even if no one person wants to e.g. summarize all the major arguments to give now vs. later, ten people can each contribute a small portion of an article and get the same result.

CEA tried to do some of this with EA Concepts, a proto-wiki with well-written articles on a range of topics. But the project was deprioritized at one point and never picked back up — mostly because it takes a lot of time and energy to create and maintain anything close to a complete list of important concepts in effective altruism.

The EA Forum seems like a better way to do this, for a few reasons.

Why host this on the EA Forum?

There have been multiple attempts to create an EA wiki before, including EA Concepts, but none have really taken off. It’s hard to get the necessary volume of volunteer work to compile a strong encyclopedia on a topic as broad and complex as effective altruism.

Building a new wiki on the EA Forum has a few advantages over using a separate website:

  • Constant attention: Hundreds of people visit the Forum every day. While tag pages don’t get many edits now, we’ll be doing a lot of work to promote them over the next few months. Anyone who visits the Forum will be prompted to contribute; if even a small number decide to help, I think we’ll have a larger collection of volunteers than any past EA wiki project.
  • Strong SEO: The effectivealtruism.org domain gets a lot of traffic, which means it tends to show up in search engines for relevant terms. Once we’ve set up a collection of heavily cross-linked wiki articles, I expect that the articles will begin to draw a lot of new visitors to the site.
  • Automatic updates through tagging: While the whole “articles are tags” thing can feel weird at times, it also means that many articles will be attached to an ever-growing list of relevant posts.
  • Professional support: The Forum is run by CEA and draws from the technical work of developers from both CEA and LessWrong. We constantly add new features, and if something breaks, we have the resources to fix it. CEA’s resources also allow us to provide incentives for Wiki editing (more news on that soon, but not in this post).

What are the next steps?

Pablo’s approach involves setting up as many relatively short articles as possible, to give editors something to work from. He started by creating ~150 “stubs”, or one-sentence articles, that he expects to develop in the coming months (he has begun to expand some already). He will continue to add more content for the foreseeable future, but all of his articles are open for editing — he plans to incorporate editors’ efforts as he goes, and appreciates the help (the project is too big for any one person). 

We’re still in the process of setting up everything we’ll need for what I plan to call “the festival” — a time of intense tagging and editing where the community gets together and really kicks things into gear. The festival will include prizes for the most helpful editors, and perhaps other treats. (Update: A previous version of this post mentioned the use of a progress bar to track tagging, but we're already at 80+% of posts with 25+ karma having tags, so there isn't much progress left to make; we'll skip the bar.)

That said, we’ll still be keeping track of edits and tagging efforts before then! If you want to help out, don’t wait for the festival — there’s a lot of good work to be done, and you can get a head start on winning a cool prize :-)

You can see all of our tags on the Tags Portal, and many of our “wiki only” articles on this page (I’m still adding the last few — check back in a day or two). 

Within the next month, we plan to have a single page which includes every tag, sorted by category (likely similar to LessWrong’s portal).

Note that we’ll be merging a lot of tags in cases where Pablo’s additions overlap with existing tags (e.g. “Meat Alternatives” and “Animal Product Alternatives”). However, we’ll combine material from both tags in the process, so good edits will be kept whatever happens.

Start your edits!

We’re really excited about the potential of the Forum’s wiki, and we hope to see lots of edits — this is a great way to make a visible contribution to the community in just a few minutes.

Some types of edits that can be valuable:

  • Adding more details about the topic
  • Adding links to other related tags
  • Adding references to material people can use to learn more about the topic

Try to take the perspective of someone who doesn’t know much about the topic and just stumbled across it. What should that person know? Can you explain that in a way that doesn’t require specialized knowledge of other EA topics? What other things might they want to read if this topic interests them?

Other ways to contribute:

  1. Tagging. This whole project will work much better if posts are reliably linked to relevant tags. This doesn’t just mean adding a tag for the first time — upvoting tags you think fit a post especially well is also helpful!
    1. An easy first step is to make sure your own posts are well-tagged.
    2. To steal from LessWrong’s excellent guide to tags:
      1. “Think of tagging as creating a curated list of material on a topic that someone interested in that topic would want to find. If a post touches on a topic but extremely tangentially, it might not be worth tagging it. If a post is low quality but technically discusses a topic, it isn’t necessary to tag it either.”
  2. Tag creation. If a term doesn’t come up as a tag when you search for it, it doesn’t exist on the Forum. If you think it should, create it!
    1. More from LessWrong: “A good heuristic is that a tag ought to have three high-quality posts, preferably written by two or more authors.”
  3. Tag suggestion. If you think some tags should be merged, notice a duplicate tag, etc., you can leave a comment on this post, or send me a private message on the Forum (if you comment on the tag itself, we may not see it for a while).
    1. Keep in mind that the wiki is very much "under construction" — if you notice something obvious that should be fixed, assume that you are correct and that we're unlikely to object. We appreciate your help!
    2. If you want to suggest that we add a tag, but don't want to create it yourself, you should share your idea on Michael Aird's tag suggestion post, where others might see it and offer feedback.

Any questions?

If you have ideas for features that could improve the tagging system or the Wiki, please let us know in the comments.

Note that you can see a further-along version of the system at LessWrong (FAQ). If you see something on that site, assume it’s possible for us, though we may not have it set up yet.

123

59 comments, sorted by Highlighting new comments since Today at 2:49 AM
New Comment

This sounds very interesting and promising! I'm glad Pablo, CEA, and the LessWrong team are doing this work, and hope a bunch of site users will also (continue to) help out as you suggest.

I suspect some readers of this post might be interested in previous discussion about or attempts to create EA-related wiki-like things. If so, they can find some at the Wikis tag.

In-page search for pages to link to

Ideally it would be really easy to link to other pages in this EA wiki. I think the normal nomenclature is [[page]] and if there were a search feature as well, it would lead to everything being more networked. 

Actually, now you mention it, I'd really like this for regular Forum posts and (especially) comments too; I'd like to be able to type "[[" and then start typing the name of the post, automatically see a drop down menu with posts that include that text in their name, and then just click on it to have the name appear in the text, with the right hyperlink. At the moment, I have to open a new tab, find the post, and either copy the title and then separately the link or write title and copy the link. I do a lot of linking to other Forum posts from within Forum posts and comments, so this is a little annoying.

I think ideally this would also show LessWrong posts, since people link to them fairly often too. 

I say "(especially) comments" because I almost always draft posts in Google docs, where this feature wouldn't really help me, whereas I write comments in the Forum editor from the start.

  1. Why can’t the existing content from EA Concepts be used to seed the new Wiki?
  2. Are you planning to use prizes or other incentives after the “festival” is over? If not, how do you plan to handle the ongoing (and presumably increasing) maintenance burden? Do you have a (very rough) estimate for how much volunteer time will be required once the Wiki is up and running?
  3. Why is a dedicated EA Wiki better than adding EA content/perspectives to Wikipedia? I think using the main Wikipedia would have numerous advantages: easier to make incremental progress, seen by more people, more contextualized, many more languages, forces EA to interact with opposing ideas, larger volunteer pool, etc. 

Why is a dedicated EA Wiki better than adding EA content/perspectives to Wikipedia?

That seems like an interesting question. I'm wondering if one reason to use a separate Wiki is that some EA-relevant topics might not meet Wikipedia's notability requirements (i.e. couldn't get their own article there).

My guess is that’s ~90% feature, 10% bug. I think most of the value of an EA Wiki would come from content that’s both EA relevant and notable by Wikipedia’s standards, and that the EA Wiki content most likely to go stale would be that which didn’t meet external notability standards.

More importantly, if you want to get EA ideas into the mainstream, at some point you're going to have to convince people outside EA that those ideas are notable.  Wikipedia seems as good a place to do so as any, since it has established procedures for assessing new content and the payoff for success is getting EA ideas included in the world's most accessible repository of knowledge.

My guess is that’s ~90% feature, 10% bug. I think most of the value of an EA Wiki would come from content that’s both EA relevant and notable by Wikipedia’s standards, and that the EA Wiki content most likely to go stale would be that which didn’t meet external notability standards.

How familiar are you with Wikipedia? I ask because I've been an editor for 17 years (though only active at certain periods) and my sense is that Wikipedia's notability standards would be a very poor criterion to judge whether the EA Wiki should have an entry on a given topic. Such standards are in line with a "deletionist" editing philosophy for which I don't see a plausible justification (see Gwern's In Defense of Inclusionism for discussion) [I now see that Aaron had already linked to this essay]. Furthermore, even if those standards were justified for Wikipedia, I don't think they would be for a specialized wiki, which is meant to be of interest to a much narrower audience and whose criteria for inclusion should reflect this specialist focus.

More importantly, if you want to get EA ideas into the mainstream, at some point you're going to have to convince people outside EA that those ideas are notable. Wikipedia seems as good a place to do so as any, since it has established procedures for assessing new content and the payoff for success is getting EA ideas included in the world's most accessible repository of knowledge.

I'm not sure I'd describe our goals as getting EA ideas into the mainstream; I think there's some reasonable disagreement within the community concerning how much and how fast we want EA to grow. In any case, I do not agree that the most effective way of convincing outsiders that EA ideas are notable is to focus our energies on writing content directly on Wikipedia rather than on a separate wiki. Before embarking on this project, I tried that approach for a few months and felt pretty disappointed with the outcome. It was in part on the basis of that experience that I concluded this project should be done as a separate wiki.

To be clear: I think contributing to Wikipedia is extremely valuable—indeed, at the current margin I think one of the most impactful things a competent and dedicated EA could do is to work full time on improving the quality and coverage of EA content on Wikipedia, and I encourage anyone who thinks they may be a good fit for this to seriously consider this option (I'd also be happy to share my experience as an editor, so feel free to contact me privately to schedule a chat). My disagreement with you concerns not the value of contributing to Wikipedia, but the claim that Wikipedia is the best place currently to host the project of creating a comprehensive EA encyclopedia.

Thanks for this thoughtful and informative response Pablo! I've consolidate my responses to a few comments including yours here.

Thanks! Responded there.

I'm somewhat sympathetic to the points you make in the first paragraph, though I don't think they will apply universally. E.g. I would expect that"hinge of history" or "patient philanthropy" are both relatively unlikely to go stale in an EA Wiki and won't meet Wikipedia's notability criteria. (Though not sure, I'm not that familiar with these criteria.)

I feel less compelled by your second paragraph. I would guess that most of the actual work to get concepts into the mainstream, establish notability etc., will need to be done outside of Wikipedia: e.g. actually founding a nonprofit, publishing a paper, and getting established media to write about you. So by the time you even have a chance to convince other Wikipedia editors that some topic meets their notability criteria, a lot of work has already been done - and the community doing that work may well be able to make good use of a Wiki that can already support them while doing that work.

"hinge of history" or "patient philanthropy"

Those concepts definitely fail to meet Wikipedia's notability standards, and I think they are a good example of why those standards are inadequate for the EA Wiki.

Thanks for raising these issues Max! I've consolidate my responses to a few comments including yours here.

As you stated, there are some advantages to running this project through Wikipedia:

  • Bigger audience
  • Greater context/longer articles that dig into more topics
  • Many articles exist in other languages
  • It has formatting options that the Forum doesn't (yet)
  • Contributions from people outside the EA community, including viewpoints that wouldn't be as likely to come from Forum readers

However, there's no reason we can't leverage a lot of this with the Forum's wiki. In many or even most cases, I'd expect that the "further resources" section of an article will include a link to the topic's Wikipedia page (or some other detailed resource). 

And I'd hope that the people who already work on EA-related Wikipedia pages will keep doing so; I agree with you that this seems really valuable for helping to make EA ideas more mainstream. 

However, there are some issues with trying to run everything through Wikipedia // benefits to making a wiki here:

  • As Max said, there are notability concerns about a lot of potentially good content. And my impression is that over time, Wikipedia has become progressively more strict about what qualifies as "notable" (see this Gwern essay, though perhaps things have gotten much better since 2009).
  • It hamstrings our ability to pay for the public good of useful EA content. Vipul Naik used to sponsor the creation of EA-related Wikipedia content, but his top editor got banned and Wikipedia's mods froze the whole thing. If CEA wants to sponsor articles about EA topics that we want people to read, we need something like EA Concepts or the EA Forum wiki (at least, that's my understanding — can you think of ways around this that aren't just clear violations of Wikipedia's rules?).
    • We could ask volunteers to do this, but it seems really good to have access to both options, and to be able to pay people for really good work.
  • Wikipedia can be an unwelcoming and intimidating place to make edits, especially for newcomers (I've heard this from many people). The Forum has many fewer rules/regulations and (I'd hope) somewhat friendlier mods, who are accountable to the broader EA community (Wikipedia's editors aren't accountable in this way).
  • CEA and LessWrong both have developers available to add features to the Forum's wiki; we can be more flexible in adapting to things that are useful for our readers than Wikipedia is (on the other hand, Wikipedia has a much bigger team, and may add features we can't replicate at some point, so this feels like a toss-up).
  • Wikipedia's articles are often going to be longer/more unfocused than would be ideal for someone trying to get up to speed with EA ideas. For example, an article about "Russia" on our wiki is likely to end up being more relevant to our community members than any article about Russia I can imagine Wikipedia allowing.
    • This doesn't mean people should only read about Russia on the Forum — certainly not! — but I think there's value in having a "Russia through an EA lens" page for people to look at alongside the Internet's many other sources of information about Russia. Same goes for lots of other topics.
  • If the Forum wiki project goes well and we end up with a lot of good, well-sourced articles, that makes it easier for volunteers to edit relevant Wikipedia pages.

Thanks Aaron! Consolidating my replies to a few different comments here.

I think the notability concerns are real, and greater than I’d originally thought. Pablo has lots of experience as a Wikipedia editor and I don’t, so I’ll defer to him. And it does seem quite telling that Pablo originally tried the Wikipedia approach for a “few months and felt pretty disappointed with the outcome.” 

That said, I’m still pretty sure there’s a lot of low-hanging fruit in terms of content that’s notable, EA relevant, and not on Wikipedia. “Longtermism” is a good example. I also suspect (though more experienced Wikipedia people should weigh in) that if we made good progress on the low hanging fruit, terms like “hinge of history” and “patient philanthropy” would be perceived as considerably more notable than they are now. (FWIW, I also think that having tags for things like “hinge of history” is a perfectly reasonable Minimum Viable Product alternative to a dedicated Wiki.)

To clarify my position, I do think a dedicated EA Wiki would be extremely valuable. But I think there’s a significant chance that a dedicated EA Wiki won’t be completed and/or maintained. That’s what’s happened to multiple previous efforts to build an EA Wiki, so that’s my baseline assumption unless I see a plan that’s obviously thought very long and hard about sustainability. I certainly don’t get that impression about this plan given Aaron’s comment about what it would take to keep the site running:

I don't have a great estimate for how much volunteer time we'd need to keep things running, but I'd expect the bare minimum to be less time than Pablo and I are putting in, such that further volunteer contributions are a nice addition rather than an existential necessity. If we were volunteer-only... maybe 15-20 hours per month? That's enough time for a few dozen minor edits plus a couple of substantive new articles.

I’m very confident that estimate is well short of the time required to upkeep a dedicated Wiki. While I don’t have experience as a Wikipedia editor, I have quite a bit of experience with a previous employer’s internal Wiki. It was immensely valuable. It was also immensely difficult to develop and maintain. There were countless emails and meetings on the theme “we need to clean up the Wiki (and this time we really mean it!)”, and in my experience it takes a combination of that and making the Wiki part of peoples job responsibilities/evaluations to maintain something useful. It’s amazing how quickly information gets stale, and it gets harder to keep things updated the as you get more entries in your Wiki. If you build an EA Wiki that gets to the level of having a page for “Russia” (to use Aaron’s example), you’re going to need a lot more volunteer (and/or paid staff) time than a few days of someone’s time a month.

I also get the sense that you’re underestimating the financial and opportunity costs of having CEA and LW developers responsible for maintaining/adding functionality, based on this comment:

CEA and LessWrong both have developers available to add features to the Forum's Wiki; we can be more flexible in adapting to things that are useful for our readers than Wikipedia is (on the other hand, Wikipedia has a much bigger team, and may add features we can't replicate at some point, so this feels like a toss-up).

Wikipedia already has more features (including features that would be valuable for an EA Wiki like translation) and a much bigger team (that doesn’t have competing priorities and doesn’t cost CEA anything), so it seems to me like CEA/LW will be constantly playing catchup. And any time they spend adding new features or even just maintaining a dedicated Wiki is time they won’t be able to work on other valuable projects.

My overarching concern is that you’re seriously underestimating the ongoing costs of this project, which will basically continue in perpetuity and increase over time. This has been the issue that sank previous attempts at an EA Wiki, and honestly it’s a pretty big red flag that you “don't have a great estimate for how much volunteer time we'd need to keep things running.” 

I’d urge you to do some more research into what the costs will look like over time (i.e. talk to people involved in previous EA Wiki attempts and people who have lots of experience with dedicated Wikis) and to think about “all in” costs as much as possible (for example, you’ll want to include the ongoing cost of finding, training, and overseeing volunteers and account for volunteer turnover). I would really love to see a dedicated EA Wiki get built and maintained, I just think that if you undertake this project you need to have a realistic picture of the ongoing costs you’ll be committing to.

Hi Anonymous,

Thank you for your thoughtful and extended feedback. I appreciate the time you have taken to raise a number of valid concerns. I will just respond to a few of your points, since much of what you say is in reply to Aaron's previous comment, and I don't want to interfere with that conversation.

I think the worry that the Wiki may fail due to insufficient contributions is very real. As you note, none of the previous attempts to build something like what we are trying to accomplish here have succeeded. And it appears that this is a common phenomenon with general efforts to create specialist wikis. Forecasting is one of my hobbies, and I'm well aware that the base rates aren't in our favor.

This was is fact my primary concern back when I was considering this project for a grant application. The reason I eventually decided to go ahead—besides feeling that I had a somewhat higher shot at success than my predecessors based on my experience editing Wikipedia and the insight this experience gave me about my capacity to feel motivated long-term by a project of this nature—was that I thought I could gain more information by just trying things out for a few months. The money costs for EA Grants were relatively modest, as were the time costs for me: at the time I didn't have any other project I felt excited about, and I don't think I would have spent those months very productively otherwise.

Fortunately, the experiment was a success: by the time the grant was over, I had not only produced more content than I had promised, but had discovered that I found writing these articles a more enjoyable experience than I had anticipated. Since then, I have continued to work on the project, and all the direct evidence indicates that lack of motivation will not be a serious impediment. This may still be insufficient to warrant an update from the naive prior to the point that I feel super confident that I will either continue to work on this project full-time for at least the next five years or find a suitable replacement, but if I had to guess, I'd estimate the chances of this happening at something like 65%.

How sensitive are your worries to scenarios in which the main paid content-writer fails to stay motivated, relative to scenarios in which the project fails because of insufficient volunteer effort? I'm inclined to believe that as long as there is someone whose full-time job is to write content for the Wiki (whether it's me or someone else), in combination with all the additional work that Aaron and the technical team are devoting to it, enough progress will probably occur to sustain growth over time and attract volunteer contributors. I'm modestly confident in this, but I'm much more confident in that it makes sense to (again) test the hypothesis experimentally, by trying to make the Wiki happen and see how excited people feel about it after a period of a year or so.

Thanks Pablo… I appreciate your thoughtful engagement with my comments, and all the hard work you’ve put into this project.

How sensitive are your worries to scenarios in which the main paid content-writer fails to stay motivated, relative to scenarios in which the project fails because of insufficient volunteer effort? I'm inclined to believe that as long as there is someone whose full-time job is to write content for the Wiki (whether it's me or someone else), in combination with all the additional work that Aaron and the technical team are devoting to it, enough progress will probably occur to sustain growth over time and attract volunteer contributors.

I’m not at all concerned that “the main paid content-writer fails to stay motivated” since that can easily be solved by finding a suitable replacement. I worry a bit about “insufficient volunteer effort”, but mostly see that as a symptom of my main concern: whether organizational commitment can be sustained. 

If CEA has a good understanding of what it will cost to create and maintain the necessary content, technical platform, and volunteer structure and commits to (indefinitely) paying those costs, I’d feel pretty optimistic about the project. I’ve expressed some concerns that CEA is underestimating those costs, but would like to let Aaron respond to those concerns as I may be underestimating the paid staff time CEA is planning or otherwise missing something.

Note on this response: I really appreciate your engagement on this! My goal for this comment is to clarify some things I didn't go into much detail on before, and better represent the way we're currently thinking about the project (as something CEA cares about a lot and will continue to care about).

That said, I’m still pretty sure there’s a lot of low-hanging fruit in terms of content that’s notable, EA relevant, and not on Wikipedia. “Longtermism” is a good example. I also suspect (though more experienced Wikipedia people should weigh in) that if we made good progress on the low hanging fruit, terms like “hinge of history” and “patient philanthropy” would be perceived as considerably more notable than they are now.

I agree with all of this. However, I think CEA would have to tread carefully to support this work without violating Wikipedia's rules about paid editing. I may think about this more in future months (right now, I'm juggling a lot of projects). If you have suggestions for what CEA could do in this area, I'd be happy to hear them.

Meanwhile, I'll declare this as clearly as I can, using bold text in lieu of cash: I am very happy to see work done on EA-relevant Wikipedia pages, and I think that such work ought to be appreciated by the community at large (and, where appropriate, considered in grant applications, job applications, etc.)

While I don’t have experience as a Wikipedia editor, I have quite a bit of experience with a previous employer’s internal Wiki. It was immensely valuable. It was also immensely difficult to develop and maintain. There were countless emails and meetings on the theme “we need to clean up the Wiki (and this time we really mean it!)”, and in my experience it takes a combination of that and making the Wiki part of peoples job responsibilities/evaluations to maintain something useful. It’s amazing how quickly information gets stale, and it gets harder to keep things updated the as you get more entries in your Wiki.

I also have experience with an employer's (valuable) internal wiki, and I appreciate this point. However, I'd expect that keeping information extremely up-to-date (e.g. making weekly updates to a large range of entries) is going to be more important for a corporate wiki than a conceptual wiki. 

My employer's internal wiki had lots of articles that were constantly becoming wrong, in ways that would impede our work if the wrongness wasn't corrected ("this is no longer the password you need", "this menu item has been renamed", etc.). On the other hand, articles like "Longtermism" or "Wild Animal Suffering", may be expanded from time to time, but it's rare that text in such an article will suddenly become wrong. 

This doesn't mean that decay isn't a concern — just that it's less of a crisis than it would be if e.g. a company were to stop making it anyone's responsibility to edit their own wiki.

If you build an EA Wiki that gets to the level of having a page for “Russia” (to use Aaron’s example), you’re going to need a lot more volunteer (and/or paid staff) time than a few days of someone’s time a month.

I think I wasn't clear enough in what I meant the "15-20 hours" to represent, and I may have come off as blasé in a way I didn't intend.

Quoting myself:

I'd expect the bare minimum to be less time than Pablo and I are putting in, such that further volunteer contributions are a nice addition rather than an existential necessity.

To clarify: I think of "bare minimum" as something like "many articles are checked on ~once  per year, and the volunteer spends a few minutes thinking about whether they want to add anything; a volunteer looks at each edit made by a Forum user and makes small fixes/reversions as needed".

With "many", I'm leaving out articles like "decision theory" and "Abhijit Banerjee", and focusing on e.g. articles about core EA orgs and active research areas.

And when I think of "volunteer time", I'm thinking of people who see themselves as "wiki volunteers", rather than the general population of the Forum — I'd also expect people on the Forum to put in quite a bit more time, because (unlike those of other EA wikis) the Forum's wiki articles will be very salient to them. 

Our tag pages got over 2000 views in each of January and February, even before most of the content uploading happened. By the time we've put in months of additional paid work and added new features to improve the wiki and make it more visible, I expect that number to increase by a lot. Only a tiny fraction of those views will turn into edits, whatever UI we employ to encourage them, but that's still a lot of additional hours.

Still, I wouldn't think of a user who makes one five-minute edit every six months as a "volunteer". So my 15-20 hour estimate didn't include this kind of activity.

The result I'd expect from a "bare minimum" outcome, combining dedicated volunteer work and other user edits: the wiki continues to be a useful resource which people refer to often, especially for its articles on evergreen concepts that don't change often. It does decay somewhat, and few new articles are written, but it remains substantial enough that a few dedicated people could jump in and resurrect it more easily than they could start a new wiki.

It's possible that 15-20 hours/month is too low even to expect this level of maintenance; as I note below, I didn't spend much time coming up with the estimate (as I don't think it will be relevant for a long time).

*****

That said, I would be very disappointed in the "bare minimum" outcome, and would go to considerable lengths to bring in more volunteer support.  I mostly set my own priorities at CEA; even if I came to believe that doing a lot of dedicated wiki work wasn't a good use of my time, and we decided to stop paying for work from Pablo or others like him, I can't imagine not wanting to spend some of my time coordinating other people to do this work.

A number I'd be happy with, where I'd expect to see the wiki grow and flourish? At 500 articles, with three hours/article/year, that's 125 hours/month — not a very confident estimate, but one that seems plausible for a good outcome. That would take a lot of coordination (if we want to have a couple dozen people putting in an hour per week*), but I'd expect to be on the front lines of that effort.

*I'd expect the actual structure of editing to look a bit different than this, because so many members of the community are going to be invested in specific articles; I'd guess that e.g. someone from 80K would make substantial edits to 80K's article every so often, and that the same would be true for many other editor/topic combinations.

My overarching concern is that you’re seriously underestimating the ongoing costs of this project, which will basically continue in perpetuity and increase over time. This has been the issue that sank previous attempts at an EA Wiki, and honestly it’s a pretty big red flag that you “don't have a great estimate for how much volunteer time we'd need to keep things running.” 

The reason I haven't spent much time thinking about the "volunteer-only" version of the wiki is that Pablo has a grant to work on this project for many months to come, and the project is also one of my highest current priorities at CEA. If it starts to seem like one or both of those things will stop being true in the foreseeable future, I expect to put a lot more time into preparing for the "volunteer-only" era.

A comparison: If you asked me "Aaron, who would take over the EA Newsletter if you got hit by a bus?", I wouldn't have a good answer on hand. That doesn't mean I think the EA Newsletter isn't important, or that it doesn't take much time to produce; I just don't expect to stop running it anytime soon, or to be hit by a bus.

Wikipedia already has more features (including features that would be valuable for an EA Wiki like translation) and a much bigger team (that doesn’t have competing priorities and doesn’t cost CEA anything), so it seems to me like CEA/LW will be constantly playing catchup. And any time they spend adding new features or even just maintaining a dedicated Wiki is time they won’t be able to work on other valuable projects.

As I've said, I really want to see people contribute to the real Wikipedia in addition to our dedicated wiki. 

But given that LessWrong already has their own wiki, which ours is a copy of, I expect them to keep adding new features to theirs which ours will adopt (this is already how the EA Forum gets most of its new features). This is how they'd be spending their time with or without us.

I'll retract my "toss-up" comment; I don't really know what I meant by it, having written that comment quickly and without editing. I do think Wikipedia will always have a better overall feature set than our Wiki — thanks for making that point clearly.

But I expect us to occasionally implement things that are a bit better than Wikipedia's version of the thing, and I also think there's a lot of value in having something that is almost as good as Wikipedia in most basic respects that we also control (rather than being at the mercy of notoriously hard-headed admins).

I think CEA would have to tread carefully to support this work without violating Wikipedia's rules about paid editing. I may think about this more in future months (right now, I'm juggling a lot of projects). If you have suggestions for what CEA could do in this area, I'd be happy to hear them.

The paid editing restrictions are a bigger issue than I’d originally realized. But I do think it would be helpful for an experienced Wikipedia editor like Pablo to write up some brief advice on how volunteers can add EA content to Wikipedia while adhering to all their rules. Sounds like Pablo has some other experiences to share as well. That plus a list of EA content that would be good to get on Wikipedia (which I believe already exists) would probably be enough to make some good progress.

But I do think it would be helpful for an experienced Wikipedia editor like Pablo to write up some brief advice on how volunteers can add EA content to Wikipedia while adhering to all their rules.

Darius Meissner and I are in the process of writing exactly such a document.

I feel like another thing that might help with causing more EAs to actually do this (as opposed to helping them do it better) is finding a way to make the impact of editing important Wikipedia articles more legible to other EAs and more likely to benefit one's status. Maybe it could be as simple as someone high-status in EA emphasising how valuable this is in a salient way (like a new top-level post, rather than a comment or an old post) and encouraging other EAs to link to their Wikipedia user profile from their EA Forum bio.

Nice!

In the meantime or in addition, some readers of this thread might be interested in some related things Brian Tomasik wrote:

(I think I've only skimmed these myself.)

Our tag pages got over 2000 views in each of January and February, even before most of the content uploading happened. 

Out of interest, do you mean all normal tag pages that can be used for tagging, or all "wiki-only tag pages", or both types of tag pages put together?

(Also, I've appreciated everyone's contributions to this discussion here.)

Thank you Aaron for this detailed engagement! 

Sounds like we’re agreed that Wikipedia editing would be beneficial, and that working on Wikipedia vs. a dedicated wiki isn’t necessarily in direct conflict.

I mostly set my own priorities at CEA; even if I came to believe that doing a lot of dedicated wiki work wasn't a good use of my time, and we decided to stop paying for work from Pablo or others like him, I can't imagine not wanting to spend some of my time coordinating other people to do this work…

The reason I haven't spent much time thinking about the "volunteer-only" version of the wiki is that Pablo has a grant to work on this project for many months to come, and the project is also one of my highest current priorities at CEA. If it starts to seem like one or both of those things will stop being true in the foreseeable future, I expect to put a lot more time into preparing for the "volunteer-only" era.

As I wrote to Pablo, my biggest concern about this project is that CEA won’t sustain a commitment to it. Pablo has a grant “for many months to come”, but what happens after that? How likely do you think it is that CEA/EA Funds will pay for Pablo or someone else to work full time on content creation for years to come? If you think that’s unlikely, then you need a realistic “volunteer-only” plan that accounts for the necessary staff, incentives, etc. to implement (and if there's not a realistic version of the "volunteer-only" plan, that's a good thing to learn ahead of time. ) In the same vein, I’d suggest giving serious thought as to the likelihood that an EA Wiki will remain “one of your highest priorities” (and/or a top priority for one of your colleagues) over a timeframe of years not months.

Honestly, a significant part of the reason I’m concerned is because I feel like accurately estimating the cost of projects (and especially the costs to keep them up and running after an initial push, including the opportunity costs of not being able to pursue new projects) has been a historical weakness of CEA’s and likely the root cause of CEA’s historical “underlying problem” of “running too many projects.”  

These are all reasonable concerns, and I agree that there are cases where CEA hasn't done this well in past years.

As soon as the wiki is up and running, and we have a sense for what "maintenance" looks like for Pablo and I (plus the level of volunteer activity we end up with after the festival), I think we'll be in a much better place to make contingency plans, and I picture us doing much of the research/planning you called for in April. (I work in a series of monthly sprints; this month's sprint is launching the wiki, and future months will involve more thinking on sustainability.)

  1. A lot of existing content from EA Concepts is being used to seed the new Wiki (I didn't mean to give the impression that this wasn't the case). However, some of Pablo's articles weren't related to EA Concepts articles, and some EA Concepts articles didn't seem like a good fit for the Wiki or have gone obsolete (e.g. articles on various evaluators' charity recommendations from 2017), so not everything made it over.
  2. I wouldn't be surprised if we continued to have additional "festival" events every so often, as a chance for people to look over lots of articles and notice things that needed updating. Aside from that, we'll have me and Pablo continuing to work on this for the foreseeable future, as well as (potentially) one or more other contractors. And I expect the greater prominence of the wiki to bring out a lot of editors naturally — existing tags have already gotten a lot of volunteer contributions, and it's really helpful to be able to say "you should throw that in the wiki for [TOPIC]" when someone shares something interesting.
    1. I don't have a great estimate for how much volunteer time we'd need to keep things running, but I'd expect the bare minimum to be less time than Pablo and I are putting in, such that further volunteer contributions are a nice addition rather than an existential necessity. If we were volunteer-only... maybe 15-20 hours per month? That's enough time for a few dozen minor edits plus a couple of substantive new articles.
  3. I'll answer this question in my reply to your second comment in this thread.

Glad to hear the EA Concept content was leveraged! 

I've consolidate my responses to a few comments including the rest of yours here.

Making a Wiki successful is always about seeding content. There's a lot of past content that could be copied over and updated, but it's not pleasant work, so it's good that Pablo has a grant.

I think it would be helpful if wiki only tags had an icon marking them as such and a programmatic way to request that the tag be unlocked and added to a post.

This is a good suggestion — I'll pass it on to the developers.

Allow every page to have a karma rank and then advanced search and ranking

Want to find the best EA books? Rank them by Karma
Want to find the EA orgs in animal advocacy? Search them by that filter. Etc etc.

What would be worthy of an up vs down? I was thinking something along this line also though, but my thought was rank them based on didactic potential according to an SNT framework - If you think it is a really important concept(S) but not many people know about it(N), and people would be interested if they did find out (T), this is the highest priority page. 

Is this what you meant by best books or were you just thinking rank them by how much you liked them?

I think anything would better than nothing. But yes ideally you could have a number of things to vote on - enjoyableness and usefulness. Or get people to estimate how much they'd have paid to read the book or something. But I think just straight up or down is a good place to begin.

Allow pages to have numerical entries 

This is a database, so it might as well have room for numerical entries. Allow pages around projects to have estimates of number of staff, spending, etc etc added and searched.

I could imagine this being a future feature! That facet of Wikipedia is well worth imitating.

Low friction editing

I'd love to see it being very easy to suggest an edit on a page, even if that edit requires approval. An edit button at the top which immediately allows you to add suggestions for instance.

Even better would be that all pages are editable all the time and that anyone can type as they read, but I doubt that will be popular (though it would generate much more editing).

All tags are already "editable all the time", other than a few we've locked as "admin only" for various reasons (e.g. the "frontpage" tag, which isn't really an article). The editing process is the same as for Forum posts — very easy!

If you're referring to posts rather than tags, that's a very different suggestion.

I think this Wiki has a decent shot at success, similar to other niche resources like SEP and Wolfram Mathworld, given the clear need, and Pablo's and engineers' efforts. And it would be super-useful if it does so.

Two current bug/feature requests:

  • It's currently very unclear where to find the Wiki content. For example, I expected to find it  here and also here. Reading this discussion (by Pablo and Michael), I expected to see the Wiki page being discussed, or at least a link to it, but I couldn't.
  • I think the idea that you're presenting Wiki pages as modified versions of "tags" is very (and fundamentally) confusing user experience (even if that's how they have to be coded). Can users not understand wiki pages as first-class entities, and say that a tag is one way that we can use a Wiki page?

It's currently very unclear where to find the Wiki content. For example, I expected to find it  here and also here. Reading this discussion (by Pablo and Michael), I expected to see the Wiki page being discussed, or at least a link to it, but I couldn't.

If what you're after is a list of all wiki-only tags (which I believe are "the wiki content" as it stands), then a partial list is available here. Aaron linked to that in the original post here: 

You can see all of our tags on the Tags Portal, and many of our “wiki only” articles on this page (I’m still adding the last few — check back in a day or two).

(If you just meant the link to the relevant Forethought page, then Aaron's reply to you here already covers that.)

It's currently very unclear where to find the Wiki content. For example, I expected to find it  here and also here. Reading this discussion (by Pablo and Michael), I expected to see the Wiki page being discussed, or at least a link to it, but I couldn't.

The "all tags" page will eventually (within the next few weeks) link to all articles, including those that are currently "wiki only". 

The wiki-only version of the tag being discussed was merged with the active tag, so the wiki-only version no longer exists (see subsequent posts in the discussion).

I think the idea that you're presenting Wiki pages as modified versions of "tags" is very (and fundamentally) confusing user experience (even if that's how they have to be coded). Can users not understand wiki pages as first-class entities, and say that a tag is one way that we can use a Wiki page?

That's how we've decided to refer to those entities internally (the wiki as a collection of articles, some of which are also tags). This decision was made over the weekend, so previous discussion of the wiki is a bit of a hodgepodge — we'll use the less confusing system in future discussions (and try to enshrine it in the UI as well).

Suggestion: Some way of displaying quantitative info about how much someone has contributed to this EA wiki

Off the top of my head, I see roughly two options for what info to display, and roughly three options for where to display this.

Options for what info to display:

  1. Just the total number of pages the user has created, edits they've made to pages, and comments they've made on the discussion/talk pages associated with wiki pages
  2. The user's "wiki karma"; a parallel system similar to the current Forum karma system but separate from that. People could upvote or downvote page additions, page edits, and comments on discussion/talk pages, and the info from that is aggregated as a wiki karma score.
    • I'm not sure if this should work exactly like a parallel version of the Forum karma system or be somewhat different (e.g., no distinction between strong and weak votes)
    • I'm not necessarily saying this should literally be called "wiki karma"; that's just what I'm calling it for now

Options for where to display this info:

  1. On each user's regular Forum profile
  2. On some dropdown or link that is accessible from each user's regular Forum profile, but not just immediately visible there
    1. E.g., each person's profile could have a link to their "wiki profile"
    2. Or maybe this dropdown or link would only appear for users who have made at least 1 wiki contribution
  3. On user profiles that are associated just with the EA wiki

For each of those options, one could also consider whether this should just be visible by default, visible by default but with the ability to opt-out from the user settings menu, or not visible by default but with the ability to opt in.

My key rationale for this suggestion is that I think a lot of EAs like feeling that their contributions are seen and appreciated, like increasing basically any quantitative target you give them, and/or like gaining status in the EA community. (I know I like these things myself.) These suggestions could help this EA wiki harness those drives in a productive way, just as I imagine the current karma system does for Forum contributions.

But the same suggestion might also be worth doing for the separate reason that it helps people decide what pages to bother reading, what edits to keep, etc., just as the current karma system helps people decide what posts and comments to read. See also a suggestion by Nathan Young.

Allow the submission, search and ranks of jobs.

There is no reason why the EA forum shouldn't become an even more effective jobs board than 80k.

80K has a deep network of contacts who send them jobs, and has a staff member who spends many hours/week keeping their job board up to date. I think it would be very difficult to find enough volunteers to replicate that effort on the wiki (especially for things that change as quickly as job openings). If I could find that many dedicated volunteers, I'd prefer to have them work on things that an org with 80K's resources isn't already doing.

I think if you give people the option to submit jobs, either they'll use it or they won't. If they do then it will be a useful resource for EA.

I think that the off moments of 1000s of eas could be at least comparable to 80k's network tbh. 

I think my prior is that people like random crowdsourcing more than you think.

The EA Job Postings Facebook group has 10,000 members  and has existed for many years. It gets something like one new posting per day (maybe a bit more?), and the average posting gets maybe five likes (the Facebook equivalent of an upvote).

I think it would take considerable effort for the Forum to get the same kind of awareness for "jobs" functionality, and even that level of awareness wouldn't necessarily come close to what 80K provides (I suspect the average person would find 80K's job board much more useful than that Facebook group). There are features we could build to improve on the Facebook group, but that's time that developers aren't spending on other features; likewise, moderating job postings to make sure they actually fit on the Forum takes time that moderators aren't spending elsewhere.

I'm not saying that the Forum couldn't provide a reasonable job board eventually — maybe we will! — it just seems like a low priority compared to a variety of things where we have no "competition" (for example, improving the articles on the wiki and creating useful features for listing and categorizing other things). You've made a lot of suggestions on this thread: work on one of those suggestions trades off against work on others.

I think you may share an instinct of mine — to say "oh, this doesn't seem hard to program" — but working for several years with the programmers behind the Forum has taught me that my instincts in this area are not reliable.

Combine EA hub profiles and EA forum profiles and then allow them to be easily tagged

 

Adding more details about the topic

Quick question to understand how I can help here: Is this about editing the texts on the tag pages themselves or about writing articles that plug gaps in the content that is covered by the articles that have a certain tag? If it’s the first, is there a way to quickly get a ranking of the most stub-like tag pages?

Thanks! I love this!

This is about editing the article pages (we're using "article" to cover everything that has a page, and "tag" for articles that can be used as tags, rather than being "wiki only").

Writing additional posts is also a good thing, of course! If you start to feel as though you are writing a post on the article page, make your content a post instead (and then you can link to it from the article page).

We don't have a ranking system or a "stub" designation yet — that's a LessWrong feature we'll be importing soon, though, at which point we'll probably be marking many articles as stubs.

Thanks for the clarification, and looking forward to the stub finder system! :-)

I expect many people currently use tags primarily as handy ways to find or share collections of posts on a given topic (I know I do), and I think that that usage is quite valuable. 

But if some wiki entries are long, then users won't immediately see the list of posts which have the associated tag. 

Users might not even be aware that there is such a list further down on the page (they might think the page is just a wiki entry). E.g., this could come up if I shared a tag page with someone as a way of saying "here are some posts you might want to check out", but they instead think I'm just sharing the wiki entry. I'm not sure how bad that'd be, since the wiki entry would probably usually also be useful to the same person, but I imagine there would be cases where it'd be less useful or interesting to them for some reason.

Do you have thoughts on how to handle this? 

Some ideas I had:

  • Make it so that, by default, only the lead section of wiki entries are immediately shown, and there's a button to expand the "box" for the entry to see more
    • Then the user would also see the list of tags relatively near the top
    • If users expand the entry box, they could click a button to collapse it again
  • Make it so that, by default, only the first certain number of words of wiki entries are immediately shown, and there's a button to expand the "box" for the entry to see more
    • Not sure what the ideal number would be
  • Something like the above options, but with the opposite default and then a button for collapsing the entry down to the lead or something
  • Giving users the option to link to the list of tagged posts (so that when you open the link, you start partway down the page rather than at the top), like how you can currently link to a specific section of a Forum post
  • Give users the option to link to the page with the entry box set to be expanded vs collapsed
    • So that people who follow the link still start from the top, but the person sharing the link can decide whether the focus is placed on the entry or on just the first part of the entry and then the list of tagged posts

We already have a "read more" label that affixes itself to long wiki entries after a certain number of (either words or characters, not sure). 

This is what I see when I open one long entry:

Do you see something else? If so, the "read more" feature might be bugged.

I think this feature means we don't need the others you suggested, though I liked suggestion #4 the most out of those.

Yeah, I see long entries that are also tags (rather than wiki-only entries) that way. 

From memory, I think I was just speculating about a possible issue without having actually seen that issue on any pages. So it seems my comment was unnecessary and roughly my first or second suggestion was already implemented!

One question I have about this is who the main target audience(s) for this wiki are? I think there are a few ways one could operationalise that question, such as:

  1. What percentage of readers do you expect/hope will be quite engaged members of the EA community, vs less engaged members, vs people who've heard of EA but never engaged, vs other people?
  2. What kinds of EAs do you expect to make the most use of this wiki?
    • E.g., people who work at EA orgs, people who don't work at EA orgs (who might therefore have a harder time getting oriented to key ideas and resources quickly without the wiki), researchers, grantmakers, policy people, other people?
  3. What kinds of non-EAs do you expect to make the most use of this wiki?
    • E.g., academics, people at think tanks, policy people, the sort of random adolescents and young adults who are predisposed to become EAs, etc.
  4. The same questions as above, but for the impact/value of the EA wiki rather than the amount of use it gets
    • It may be that a type of person who will use the wiki relatively rarely will still be the main type of person through whom the impact of the wiki flows
      • E.g., maybe most readers will be EAs, but the marginal value of the wiki will be much higher for a handful of non-EAs who otherwise wouldn't have encountered these ideas or would've encountered a less compelling or clear intro 

(Apologies if this has been addressed somewhere here already and I've just forgot that.)

A focus on user experience.

The core aim should be to make this database something that EAs actually use to find things in their daily lives. 

Use Roam.

I find Roam a fantastic database building tool and it might be good to just use it here rather than building our own.

I don't think this is a great suggestion since I think we'll want different features, but if it could be built on top of Roam, I think it would be much better. 

Tag suggestion. If you think some tags should be merged, notice a duplicate tag, etc., you can leave a comment on this post, or send me a private message on the Forum (if you comment on the tag itself, we may not see it for a while).

That sentence seems to be more about suggestions about tags than suggestions of tags. For the latter, people are also welcome to comment on this post I made last year: Propose and vote on potential tags

Or do you think that it's now better to centralise such suggestions in comments on this new post instead? I'd guess it's best for them to be centralised in one dedicated place.

(Also, of course, people should feel very free to simply make tags, and should probably do that more often than suggesting the tag. That post is just there as an option for if and when a person might want to use it.)

I've just subscribed to your post, so I'll see comments in either place. 

It seems a bit better to have a single post people can subscribe to if they want to participate in discussions of this type, so I'll change the post to recommend people use your post.

(And yes, people should also feel free to make tags if they want to see them, as I said in this post.)

(And yes, people should also feel free to make tags if they want to see them, as I said in this post.)

Yeah, to be clear, my bracketed bit was to prevent my comment accidentally giving the impression that I think people should default to suggesting rather than creating tags. I don't think your post would give that impression :) 

  1. I can't find the exact location right now, but someone on LW made a web visualization of EA academic papers - lines between papers representing citations. I was thinking something like this could be done for the forum in general with hyperlinks but it might be cooler to do it with the wiki. The thought behind it outside of just being a cool visualization is that many thoughts come in clusters and being able to visualize the thoughtspace you're in might help you breakthrough plateaus more easily and visualize how things connect within ea.
  2. more of an open question but I think its relevant to think about how atomic you make the pages, as in how much ideas are embedded/hyperlinked vs written out in full.