Great post! I second this recommendation. Because I'm gathering a lot of material in the course of working on the next edition of the EA Handbook, I'll look for opportunities to post some of that to the Forum (in cases where doing so won't overlap too much with the Handbook).
an interesting/useful term or concept is discussed but never clearly defined, or is used/defined in various somewhat different ways
I think that EA Concepts did a good job mitigating this problem regarding concepts, although I'm unsure how many people read it. In general, it seems that it would be better if these sort of collections would be in one place (e.g., like they are in EA Concepts), in addition to being scattered throughout the EA Forum. Maybe an EA wiki (which is already in the works) could solve such a problem, provided enough people use it. Or maybe EA concepts could be expanded. Or maybe we should also post collections of summaries/collections on the Forum.
I agree that having some central directory or collection of the summaries/collections would be ideal. And I think all of those suggestions for achieving that are good.
Also, I think EA Concepts is great. And I think people who haven't checked it out should, or should keep it in mind when they encounter a concept they're unfamiliar with. (Conceptually also performs a similar function. It isn't explicitly EA-focused, but it was made by EAs and covers a lot of concepts EAs like to use.) However:
This suggests that one possible solution would be for the people behind EA Concepts to crowdsource (and then vet) new entries, and/or updated versions of existing entries.
P.S. #2: There are three questions I have about how one should do the “collections”, which didn't seem necessary to include in the above post itself, but which I’d be interested in people’s thoughts on.
1. Should the collections be full posts on shortform comments?
So far, I’ve posted all my collections as “shortform” comments, rather than full posts, except one (Quotes about the long reflection). This is because most have been quite short, and pretty much just lists, so they didn’t feel substantial or refined enough to be full posts.
But shortform comments aren’t displayed as prominently upon posting or when using the search function (the search function will show the user’s name but not any snippet of the comment). So there’s a higher chance that people who would’ve been interested in the collection will miss it.
That might make sense, as people have limited attention, and these collections are arguably not especially important. But I’m unsure precisely where to draw the line. And I think it'd be nice if they were easier to find through the search function, at least (e.g., if the first few words of shortform comments were displayed when searching, or something like that).
2. Is my approach to collecting sources actually useful, given the fact that relevant papers will have reference lists?
I think the answer is "Yes", given that:
But I’m not sure about this. Maybe it actually wouldn’t be worth people’s time to make collections of sources in particular. (Although, again, the time investment can be pretty small when you’re learning about a topic anyway.)
EDIT: I'm now more convinced that collecting sources is useful, and upvotes of my collections seem to provide weak evidence in support of that.
3. Would it be better to spend the same amount of time producing a lower number of more sophisticated collections, as proper databases or something (e.g., using Mendeley?)?
Somehow I didn’t learn about these more sophisticated approaches at all during my university studies or when writing a paper, so I don’t know how those approaches work or how valuable they are, and thus I can’t really answer this. But maybe this is the approach I and/or others should be taking, either when posting these collections or just when doing research in general.
Update: The new tags system now captures some of the value "collections" can capture (which is great). Though there are two aspects* of the tag system that mean collections can still add value:
So I think there can still be value in collecting sources even on topics that now have tags. In such cases, it seems it would be best to ensure that the EA Forum posts with the relevant tag are in the relevant collection, and vice versa.
And more obviously, there can still be value in collecting sources on topics for which there isn't a tag. And perhaps if that topic later gets a tag, people can easily kick that tag off using that collection.
Tl;dr I recommend more EAs post summaries of useful ideas and collections of quotes, sources, definitions, terms, etc. Collections in particular can be fairly easy to make when you’re learning about or researching a topic anyway. And both summaries and collections can make the road easier for those who follow.
Effective altruism is still pretty new, as are several of its allied communities (e.g., the rationality community), and as are many of the topics and concepts of interest to EAs. It seems to me, and I believe to many others, that this leads to the following situations cropping up surprisingly often:
Essentially, a lot of things of interest to EAs, and a lot of work by EAs, seems to not yet be neatly consolidated and easily searchable. This makes it harder to learn from and build on the work of others. It may also lead to:
One solution to this is for people who are learning about or researching something anyway to:
I think that solution is great (as does the forum’s moderation team). And I’ve been doing some of that summarising myself.
For example, a bunch of great stuff had already been written on differential progress/intellectual progress/technological development, information hazards, and moral uncertainty. But a lot of those writings were long and academic, or took a certain prior familiarity with the topic as assumed.
When learning about those topics out of general interest, or in hopes of writing things that built on existing work, I discovered the lack (in my view) of central, accessible summaries on them. So I figured I should try to fill those gaps. I saw this as adding value simply by collecting together, in an accessible way, all the most important points (at least regarding the concepts themselves, not always all of their implications).
And I’ve found similar summary posts by other EAs really helpful myself, and would be keen to see more EAs writing summaries like that. You could set out purposely to do that, or just learn about things and wait until you happen to stumble upon an interesting topic for which no decent summary exists.
(See also “research debt” and “research distillation”.)
But writing summaries can take a decent amount of effort and time, and can be challenging. So a quite low effort, easy alternative I’ve started doing is posting “collections” of:
In each case, making and posting the collection took between almost no time and not much time, because I was already learning about the topic and collecting things for my own sake anyway.
I've also seen other EAs usefully collect:
I’d be keen to see more EAs posting such collections. It seems to me that, as with summaries, such collections should make it easier for people to learn of, learn about, and build on interesting ideas and topics.
Ideally, I’d hope that other people would comment to add to the collections, so that they can serve better as go-to lists for whatever they’re about. This makes the collections more valuable to future readers, as well as to the person who originally made the collection. To encourage this, I’ve always noted in my collections something like “I intend to add to this list over time. If you know of other relevant work, please mention it in a comment” - and indeed, a few kind souls have usefully done so.
So (I gently suggest, to those who feel this would suit them) go forth and get summarising! Build paths as you go about your travels!
And get collecting! Leave a trail of breadcrumbs where you walked, and signposts to interesting regions you haven’t had time to visit yourself!
And then hopefully those coming after you can travel these paths and visit these regions more easily. And hopefully they can race ahead, or to the side, or lay further paths and breadcrumbs and signposts. And hopefully this can all add up to a relatively low-effort way to help our community do good a little better.
It also seems to me that summarising and collecting might be able to serve as one small version of something like a “Task Y”: i.e., something which has “some or all of the following properties:
Relatedly, I'm quite keen on the idea of writing and posting short literature reviews. I suspect that when people look into a topic, they sometimes spend 10-40 hours looking into particular sub-topics, just to inform their own decision-making. It doesn't take much more time input to write your own notes in the format of a literature review, then spend a little while editing it at the end for clarity.
That's what I did with my short lit reviews on issues related to developing and training management and leadership expertise. Charity Entrepreneurship also write short reports on each of the main options that they consider in various stages of their recommendation process.
One danger of this approach is that by formatting it as a literature review you might come across as over-confident in your findings (even with lots of caveats), and you leave yourself open to accusations of low rigour.
Yeah, I think that'd be another great thing for people to do. Although I'd add that that might be especially valuable when there's no existing non-EA lit review on the topic (e.g., when the topic is somewhat obscure, or you're taking a particular angle on a topic that EAs are unusually interested in).
E.g., perhaps there's already a fairly good lit review on hiring practices in general, and you could add some value by writing a more summarised or updated version. But you might also be able to capture a lot of that value just by making a link post to that review on the forum and noting/summarising a few key points. Meanwhile, there might be no lit review that's focused on hiring practices for start-up-style nonprofits in particular, so writing that might be especially worthwhile (if that's roughly the subtopic/angle you were interested in anyway).
On the other hand, I think I would've guessed that the topic "how to create a highly impactful/disruptive research team" would be quite far from obscure, and would already have a solid, up-to-date lit review covering what EA people would want to know. But this post suggests there wasn't an existing lit review with that particular angle, and the post seemed quite interesting and useful to me. So there are probably more "gaps" than I would naively expect, and thus substantial value in lit reviews on certain topics I would naively expect are already "covered".