The EA Forum Editing Festival is over! (And has been, for a bit.)
I’ll use this post to look at how well the festival went, and to award prizes to people who were especially good contributors. (I’ve selected these people in consultation with Pablo Stafforini, who’s been spending more time on the wiki than I have.)
Overall
I’d consider the festival a qualified success.
The good
- Traffic to the wiki increased substantially
- Dozens of people applied at least one tag to a post they didn’t write; it feels like we succeeded in getting people to try this out at least once
- We gave many good older posts their first tags, making them more accessible
- The average wiki article is much more detailed and polished than it used to be
The bad
- Overall tagging rates spiked during the festival, but now seem to have gone back to pre-festival numbers
- Almost all our wiki edits came from three people, leaving us without a broad community of editors (for now)
I wish we’d seen more participation from more people, but I still wound up with a list of a dozen users whose work I’m happy to reward. (And I appreciate everyone who applied so much as one tag — see the “Honoring all contributors” section at the end.)
Traffic
Between 1 January and 24 February, our tag/wiki pages got ~4,400 total views, with an average view duration of 56 seconds.
Between 1 April and 24 May, those pages got ~16,700 views, with an average duration of 81 seconds.
To give a specific example: “Longtermism” is our most-visited tag, and went from 118 views (1/1 - 2/24) to 253 views (4/1 - 5/24).
One factor behind this is a test we ran in April to reroute clicks on certain EA Concepts pages to their wiki equivalents (as we might eventually sunset EA Concepts in favor of a set of wiki links). But this contributes only a small fraction of the change; between these two periods, EA Concepts lost ~4,000 pageviews, while the wiki gained ~12,000.
Tagging
As we requested, people tagged every post with 25 karma or more, then almost all posts with 20-24 karma (there are only three untagged posts in that category as I write this).
We also got a lot of strong tag suggestions from users.
Finally, the overall tag rate went up quite a bit during the festival; in the first week of May, we averaged around 100 tags per day, roughly tripling our number from the first week of April. Since the end of the festival, the rate has dropped back to around where it was at the beginning, which isn’t what I’d hoped for — but we did tag a lot of archival material in the meantime. If only new posts are getting tags at this point, each of them should get 3-4 tags on average at current rates, which seems very reasonable.
Wiki Edits
I’ve been happy to see the overall speed of progress on the wiki, but disappointed to see almost all of that progress come from three users so far (Pablo, his assistant Leonardo, and Michael Aird).
I think the initial collection of wiki content is shaping up well, but I still want us to have a broad set of editors so that new resources, papers, etc., routinely find their way into the appropriate articles. I’ll also note that Pablo, in the course of researching other community wikis, has found that “lack of contributors” appears to be the main reason these wikis fail to thrive.
Over the next few months, Pablo and I will be looking for ways to incentivize wiki edits from the wider community. A few things that come to mind:
- A feature allowing people to see their wiki edits on their own profiles
- An addition to the Forum Prize. awarding one or more wiki edits each month
- More targeted solicitation toward Forum users we expect will be strong editors for specific articles
We also have a feature in the pipeline that will allow users to vote on edits, which should be helpful.
Prizes
As promised, we’ll be awarding at least $5000 in prizes to at least a dozen users (in the form of donations to charities listed on EA Funds).
Michael Aird: $2000
Michael has contributed more words to the wiki than any author aside from Pablo, who has a full-time grant for wiki work. He also applied more than 500 tags over the course of the festival. And his meta-post on tag suggestions has also been a useful organizational tool. Finally, he’s had many conversations with Pablo and I about the details of the wiki system, and has done much to influence our policies. I don’t know where he finds the time, but I’m grateful to have him as a super-user.
Maria Shakhova: $750
Maria applied more than 800 tags over the course of the festival, including for a number of older and lower-karma posts that are now less likely to be forgotten. (As I did the research for this post, Maria’s tags gave me some good reading material!)
It’s especially impressive to me that this came from someone who’s written only a single comment during her time on the Forum! I’m glad that the Festival gave Maria this chance to contribute, and that I had the chance to see how much she’s been doing.
Tessa Alexanian: $750
When she isn’t winning Forum prizes, Tessa has been a prolific participant, with more than 350 tags applied over the course of the festival. She also created many new tags:
- Electoral politics
- Fellowships and internships
- Human Compatible
- Microfinance
- Postmortems and retrospectives
- Video
Cullen O’Keefe: $200
Cullen added more than 75 tags and created two new tags — donation matching and EA Giving Tuesday.
evelynciara: $200
evelynciara didn’t just participate actively in the festival (150+ tags, more than almost anyone) — she's also done lots of tagging in the last few weeks. Those recent tags don’t “count” towards this award, but I still love to see that the event helped to make Forum infrastructure into a habit for certain users.
Marisa Jurczyk: $200
Marisa did a bit of tagging, and also created or heavily updated three useful tags:
Vaidehi Agarwalla: $200
Vaidehi applied a few tags, but her main contribution came from new tags she created:
- https://forum.effectivealtruism.org/tag/field-building
- https://forum.effectivealtruism.org/tag/intervention-evaluation
- https://forum.effectivealtruism.org/tag/network-building
David Bernard: $125
David added more than 75 tags during the festival.
Edo Arad: $125
As a moderator, Edo winds up applying a fair number of tags (100+ during the festival) — but because he’s a volunteer and his job doesn’t actually require that he tag anything, I still want to reward his work as one of the Forum’s most active taggers.
Finn Hambly: $125
Finn applied 70+ tags, with a focus on AI-related topics.
Harrison Durland: $125
Harrison created two tags:
Both have grown on me, especially “competitive debating” — it’s nice to see another activity that seems like a natural pipeline for EA-oriented people.
nil: $125
nil created two tags:
- Organisation for the Prevention of Intense Suffering
- A tag for David Pearce — which, while it was removed from the wiki due to notability considerations, was a well-written, well-meaning contribution.
I thought the conversation around the David Pearce tag was valuable, and I’m glad nil’s work prompted it to happen.
Stefan Schubert: $125
Stefan added useful content to bibliographies on three different tags — and these were three different articles, not just the same article being forced into multiple tags. This kind of contribution is one I’d love to see more of — if you have a favorite book or paper on an EA topic, make sure it’s on the wiki page for that topic!
Honoring all contributors
As promised, we’ll use this post to remember all those who tagged any post, or added any new wiki content. Thank you all for your contributions!
If you took part and don’t see your name here, please let me know.
Those who helped
- aaronmayer
- Abby Hoskin
- abrahamrowe
- Adam Binks
- AndreaSR
- Andy_Schultz
- Animal Ask
- arushigupta
- ashnaix
- BarryGrimes
- Ben_Kuhn
- ben.smith
- BrianTan
- calebp
- Cameron_Meyer_Shorb
- Casey Riordan
- Charles_Dillon
- Charlotte
- Chi
- Cillian Crosson
- ClareDonaldson
- Cullen_OKeefe
- D0TheMath
- Dale
- DannyBressler
- David_Moss
- DavidBernard
- DavidZhang
- dehraduncallgirl
- Denis Drescher
- doktercito
- EdoArad
- elliottthornley
- Erich_Grunewald
- escorthyderabad
- ethankennerly
- evelynciara
- farrah.dingal
- FeelingSentient
- Felix Rudfeldt
- finnhambly
- FJehn
- florian-z
- freedomandutility
- FrostedSapling
- GidonKadosh
- Gk
- GraceSophie
- greg.gianopoulos
- gruban
- gunnar_v
- Guy Raveh
- Habryka
- Hamish Huggard
- Harrison D
- Harry_Taussig
- Heidi_Basarab
- helpinproject2020
- henrith
- Holly_Elmore
- Hypatia
- iamef
- ImmaSix
- jablevine
- Jack
- James Smith
- JamesOz
- Jan-WillemvanPutten
- Jason Schukraft
- Jeffray_Behr
- JoanGass
- JoanMM
- Joey
- johl@umich.edu
- JP Addison
- Jsevillamol
- JSOD
- Julia_Wise
- jwagstaff
- KarolinaSarek
- KathrynMecrow
- konrad
- kvadieselgenerator
- Larks
- leonoraahla
- Locke_USA
- LuciaC
- Lucius_Caviola
- lukefreeman
- Mahendra Prasad
- Maria Shakhova
- MarisaJurczyk
- mariushobbhahn
- Martine Klock Fleten
- Matt Boyd
- Max_Daniel
- MaxDalton
- MaximeCdS
- McKenna_Fitzgerald
- meerpirat
- Meg
- Megan Kinniment
- Michael Huang
- MichaelA
- MichaelPlant
- MichaelStJules
- Michelle_Hutchinson
- Miranda_Zhang
- MorHanany
- nehakauxesis
- Neil_Dullaghan
- new_staffer
- newptcai
- nil
- NunoSempere
- Pablo
- Peter_Hurford
- ProbablyFaiz
- Quinn McHugh
- rsturrock
- SamiM
- saulius
- shabaz
- SiebeRozendal
- simonfriederich
- smountjoy
- spreadlove5683
- tamgent
- Teo Ajantaival
- tessa
- Timdrave
- Ula
- vaidehi_agarwalla
- velutvulpes
- Vicky Clayton
- Wild_Animal_Initiative
- WilliamKiely
- WillNickols
- wuschel
- yiyang
- zdgroff
Thanks for this post :)
One thing I've been wondering about is how much impact will result (indirectly) from the wiki/tagging system, and, relatedly, from various types of individual contributions to it.
I'm wondering this both to work out whether I'm spending more time contributing than I really should and to work out whether I should do more/less encouraging of other people to contribute. (This also seems relevant for "targeted solicitation toward Forum users we expect will be strong editors for specific articles" - having a clearer impact assessment would help both in deciding when it's worth people spending their time this way and in convincing them of that.)
Currently I - as you can probably tell - feel pretty excited about the wiki and tagging system, often send people links to entries, often suggest they edit or tag things, etc. But I don't actually have a clear sense of how good a use of people's time this is.
---
I think it'd help to create an explicit theory of change for the wiki and tagging system, or maybe one for each of those two components. Maybe ideally in diagram form. Then you/we could try to use this to identify assumptions, identify what proxies we should pay attention to and date we should gather, identify what forecasts we should make, etc. (See also.)
---
Here are some other proxy/impact assessment ideas, though they're kind-of putting the cart before the horse without having worked out a more explicit theory of change first. (This also overlaps with discussion we've previously had non-publicly, where Pablo mentioned some other ideas I won't mention here.)
A caveat I should add: The wiki could be seen more as a product or website than a research output/project. And I'm more used to thinking about how to assess the impact of the latter (though I'm not an expert on that anyway).So maybe my suggestions are a bit off, and maybe it'd be useful to think more about things like:
These ideas seem good to me. I think another low-hanging fruit might be some way of displaying quantitative info about how much someone has contributed to this EA wiki, as is done on Wikipedia; see here for elaboration.
But it might be best to do all of these things after getting more sense of how much impact contributing to the wiki/tags has (see my other comment), since that'll inform how much time it'd actually be good to get people spending contributing to the wiki/tags.
We definitely want to add wiki contribution data to user profiles (it's a natural accompaniment to the lists of users' posts and comments), though the timeline for that project isn't yet established.
Oh wow, that's a fairly chunky prize! (For readers who don't follow the link to the festival announcement post, it seems worth noting that these prizes are "In the form of donations to an EA Funds-eligible charity of your choice", rather than e.g. deposits to my own bank account)
This seems like a good prompt to revisit and share my rough, interim notes-to-self on where I might donate in 2021 and why. (I plan to do more thinking and share more details around November, like I did last year.)
It seems that Effective Thesis and APPGFG aren't available via the EA Funds platform, even though both have been funded by EA Funds before? Can they be added to the platform relatively easily? (That isn't a super big deal just for this particularly $2000 donation - I'm ok just picking between LTFF, GCRI, and ALLFED - but it seems like that'd be good for other donors too.)
Once I find out whether Effective Thesis and APPGFG can be options for me for this $2k, I'll decide on the resulting shortlist.
I've added this detail to the post for clarity. Thanks!
I'll check in with EA Funds about whether they have any plans to add those options.
Thanks! (On both parts, but especially the latter.)
I haven't read this post and comment in full, but here are our criteria for adding charities to EA Funds.