I really like the general class of improving community epistemics :)

That being said, I feel pretty pessimistic about having dedicated "community builders" come in to create good institutions that would then improve the epistemics of the field: in my experience, most such attempts fail, because they don't actually solve a problem in a way that works for the people in the field (and in addition, they "poison the well", in that it makes it harder for someone else to build an actually-functioning version of the solution, because everyone in the field now expects it to fail and so doesn't buy in to it).

I feel much better about people within the field figuring out ways to improve the epistemics of the community they're in, trialing them out themselves, and if they seem to work well only then attempting to formalize them into an institution.

Take me as an example. I've done a lot of work that could be characterized as "trying to improve the epistemics of a community", such as:

• The Alignment Newsletter
• An AI alignment literature review
• Clarifying key hypotheses that alignment researchers disagree about (mostly an advisory role)
• Questioning the case for AI risk (1, 2)
• Writing community building retrospectives
• Organizing calls for local EA group organizers to share best practices

The first five couldn't have been done by a person without the relevant expertise (in AI alignment for the first four, and in EA group organizing for the fifth). If they were trying to build institutions that would lead to any of these six things happening, I think they might have succeeded, but it probably would have taken multiple years, as opposed to it taking ~a month each for me. (Here I'm assuming that an institution is "built" once it operates through the effort of people within the field, with no or very little ongoing effort from the person who started the institution.) It's just quite hard to build institutions for a field without significant buy-in from people in the field, and creating that buy-in is hard.

I think people who find the general approach in this post interesting should probably be becoming very knowledgeable about a particular field (both the technical contents of the field, as well as the landscape of people who work on it), and then trying to improve the field from within.

It's also of course fine to think of ideas for better institutions and pitch them to people in the field; what I want to avoid is coming up with a clever idea and then trying to cause it to exist without already having a lot of buy in from people in the field.

Roam Research is

> starting a fellowship program where we are giving grants to researchers to explore the space of Tools for Thought, Collective Intelligence, Augmenting The Human Intellect.

They recently raised $9M at a $200M seed evaluation and previously received two grants from EA LTFF.

Thanks, interesting.

1) One distinction one might want to make is between better versions of previous institutions and truly novel epistemic institutions. E.g. Global Priorities Institutes and Future of Humanity Institute are examples of the former - university research institutes isn't a novel institution. Other examples could be better expert surveys (that already exists), better data presentation, etc. My sense is that some people who think about better institutions are too focused on entirely new institutions, while neglecting better versions of existing institutions. Building something entirely novel is often very hard, whereas it's easier to build a new version of an existing institution.

2) One fallacy people who design new institutions often make is that they overestimate the amount of work people want to put into their schemes. E.g. suggested new institutions like post-publication peer review and some forms of prediction institutions suffer from the fact that people don't want to invest the time in them that they need. I think that's a key consideration that's often forgotten. This may be a particular problem for certain complex decentralised institutions, which depend on freely operating individuals (i.e. whom you don't employ full-time) either voluntarily or for profit investing time in your institution. Such decentralised institutions can be theoretically attractive, and I think there is a risk that people get nerd-sniped into putting more time into theorising about some such institutions than they're worth. By contrast, I'm more generally positive about professional institutions who employ people full-time (e.g. university departments). But obviously each suggestion should be evaluated on its own merits.

3) With regards to "norms and folkways", there is a discussion in economics and the other social sciences about the relative importance of "culture" and (formal) institutions for economic growth and other desirable developments. My view is that culture and norms are often under-rated relative to formal institutions. The EA community has developed a set of epistemic norms and an epistemic culture which is by and large pretty good. In fact, it seems we didn't develop too many formal institutions that are as valuable as those norms and that culture. That seems to me a reason to think more about how to foster better norms and a better culture, both within the EA community, and outside it.

Institutions for exchanging information (especially research) also seem helpful to me. For instance, many researchers circulate their work in semi-private google docs but only publish some of their work academically or on the Forum. (Sometimes, this is because of information hazards, but only rarely.) This makes it harder for new or less well-networked researchers to get up to speed with existing work. It also doesn't scale well as the community grows. It would be great if there were easy ways to make content public more easily. Wei Dai made a suggestion in this direction, and I bet there are further ways of making this happen.

This is a great post and I, like @rohinmshah, feel that simply the introduction of this general class of discussion is of value to the community.

With respect to expert surveys, I am somewhat surprised that there isn't someone in the EA community already pursuing this avenue in earnest. I think that it's firmly within the wheelhouse of the community's larger knowledge-building project to conduct something like the IGM experts panel across a variety of fields. I think, first, that this sort of thing is direly needed in the world at large and could have considerable direct positive effects, but secondly that it could have a number of virtues for the EA community:

• Improve efficiency of additional research: Knowing what the expert consensus is on a given topic will save some nontrivial percentage of time when starting a literature review, and help researchers contextualize papers that they find over the course of the review. Expert consensus is a good starting place for a lit review, and surveys will save time and reduce uncertainty in that phase.
• Let EAs know where we stand relative to the expert consensus: when we explore topics like growth as a cause area, we need to be able to (1) have a quick reference to the expert consensus at vital pivots in a conversation (e.g. do structural adjustments work?) and (2) identify with certainty where EA views might depart from the consensus.
• Provide a basis for argument to policymakers and philanthropists: Appeals to authority are powerful persuasive mechanisms outside the EA community. Being able to fall back on expert consensus in any range of issues can be a powerful obstacle or motivator, depending on the issue. Here's an example: governments around the world continue to locally relitigate conversations about the degree to which electronic voting is safe, desirable, secure or feasible. Security researchers have a pretty solid consensus on these questions-- that consensus should be available to these governments and those of us who seek to influence them.
• Demonstrate to those outside the community that EAs are directly linked to the mainstream research community: This is a legitimacy issue: regardless of whether the EA community ends up being broader or narrower, we are often insisting to some degree on a new way of doing things: we need to be able to demonstrate to newcomers and outsiders that we are not simply starting from scratch.
• Establish continued relationships with experts across a variety of fields: Repeated deployment of these expert surveys affords opportunities for contact with experts who can be integrated into projects, sought for advice, or deployed (in the best case scenario) as voices on behalf of sensible policies or interventions.
• Identify funding opportunities for further research or for novel epistemic avenues like the adversarial collaborations mentioned in the initial post: Expert surveys will reveal areas where there is no consensus. Although consensus can be and sometimes is wrong, areas where there is considerable disagreement seem like obvious avenues for further exploration. Where issues have a direct bearing on human wellbeing, uncovering a relative lack of conclusive research seems like a cause area in and of itself.
• Finally, the question-finding and -constructing process is itself an important activity that requires expert input. Identifying the key questions to ask experts is itself very important research, and can result in constructive engagements with experts and others.

On expert surveys, I would personally like to see more institutionalized surveys of key considerations like these: https://www.stafforini.com/blog/what_i_believe/ One interesting aspect could be to see in which areas agreement / disagreement is largest.

This is great! I find this extremely important, and I agree that we have a lot of room to improve. Thank you for the clear explanation and the great suggestions.

Further ideas:

1. A global research agenda / roadmap.
2. Bounties for specific requests.
   1. Perhaps someone can set a (capped) 1-1 matching for individual requesters.
   2. Better, give established researchers or organizations credit to use for their requests.
3. A peer review mechanism in the forum. A concrete suggestion:
   1. Users submitting a "research post" can request peer review, which is displayed in the post [a big blue "waiting for reviewers"].
   2. Reviewer volunteers to review, and present their qualifications (and statement of no conflict of interest) to a dedicated board that consists of "EA experts", which can approve them for review.
   3. There are strict-ish guidelines on what is expected from a good post, and a guide for reviewers.
   4. The reviewers submit their review anonymously and publicly.
   5. They can accept the post [a big green "peer reviewed"].
   6. They can also ask to fix some errors and improve clarity [a big yellow "in revision"].
   7. They can decide that it is just not good enough or irrelevant [a big red "rejected"].
4. (The above is problematic in several ways. The reviewer is not randomized, so there is inherent bias. The incentive for reviewing is not clear. It can be tough to be rejected..)
5. Better norms for linking to previous research and asking for it. Better norms for suitable exposition. These norms don't have to be strict on "non-research" posts.
6. The forum itself can contain many further innovations (Good luck, JP!):
   1. Polls and embedded prediction tools.
   2. Community editable wiki posts.
   3. Suggested templates.
   4. Automated suggestion for related posts while editing (like in stackexchange).
   5. An EA tag on lesswrong/alignment forum (or vice versa) with which posts can be displayed on both sites (like the LW/AF workflow).
   6. A mechanism for highlighting and commenting like in Medium. (Not sure I like it)
   7. Suggestions that appear (only) to the editor like in google docs.
   8. There are some great stuff already on their way also :)
7. Regarding a wiki, Viktor Petukhov wrote a post about it with some discussion following it on the post and in private communication.
8. More research mentorships. Better support for researchers at the start of their path.
9. Better expository and introductory materials, and guides to the literature.
10. Better norm and infrastructure for partnering.
11. A supportive infrastructure to coordinate projects globally, between communities. This can allow more easily to set up large scale, volunteer-led projects for better epistemic institutions. The importance of local communities here is as a vetting mechanism.

Effective altruism wiki: Intuitively, this makes a lot of sense as a means of organizing knowledge of a particular community. Also, if the US Intelligence Community is doing it, it has to be good. I know that there have been attempts at this (e.g., arbital, priority.wiki, EAWiki). Unfortunately, these didn’t catch on as much as would be necessary to create a lot of value. Perhaps there are still ways of pulling this off though. See here and here for recent discussions.

In addition to the wikis, there are also EA Concepts and the LessWrong Wiki, which have similar roles.

Two hypotheses for why these encyclopedias didn't catch on so far:

• Lack of coordination: Existing projects seemed to focus on content but not quality standards, editing/moderation, etc. Projects weren't maintained long-term. It probably wasn't sufficiently clear how new volunteers could best contribute. Resources were split between multiple projects.
• Perhaps EA is still too small. Most communities with successful wikis have fairly large communities.

Personally, I'd be very excited about a better-coordinated and better-edited EA concepts/wiki. (I know of someone who is planning to work on this.)

Building such institutions is a form of community-building. Arguably, this is one of the most important ways of making a difference since it offers a lot of leverage. It came second in the Leaders Forum survey.

(Not very important.) Hm, which result of the survey do you mean? I can't remember being given that option and can't find it immediately in that post.

Related: the term I've been using lately in thinking about this sort of thing is epistemic public goods, which I think was prompted when I saw Julia Galef tweet about the "epistemic commons". 

There might be coordination platforms around platforms such as a wiki. It’s only worth it for an individual to participate if enough other people participate. Prizes, participation by respected members, and consistently making the case for the institution might help here.

One other idea I'm aware of for solving these coordination problems is this Facebook group "for proposing crowdfunding or coordinated actions" by rationalists/EAs.

E.g.:

Cause Prioritization Wiki

Goal: Improve https://causeprioritization.org/

Commitment: Make X major edits within 3 months after the threshold (ie. creating or significantly improving a page's section)

Alternative: You can pledge money that will be used to pay editors. I'll assume a rough conversion rate of 15 USD per major edit (I would take care of dispatching the work).

Threshold: 100 major edits (equivalent) committed

Extended threshold: I will check with Issa Rice if I can move the wiki to the MediaWiki platform if at least 200 major edits get committed.

Why it needs coordination: It will only become a Schelling point to read and document information if a sufficient amount of people are already using the platform as such.

I'm not necessarily saying that that's a good solution - I haven't really looked into or thought about it - just sharing that it exists. (I think I'd also be concerned about choosing one wiki to coordinate on without first having a big discussion about which wiki it should be - but I haven't looked into the latest status of such discussions either.)