After collecting feedback from the community, I’ve decided to make June the last month that CEA will definitely award the EA Forum Prize in its current form. This means that we will publish prize posts for May 2020 and June 2020, but won’t necessarily award prizes after that. (This was also announced in the most recent prize post.)
Overall, while some users reported finding the Prize valuable or motivating, that number wasn’t quite as high as I had been hoping for. I also heard some good suggestions for ways the Prize could be restructured.
This thread is meant to collect additional suggestions for how we could run the Forum Prize — or otherwise use the funding (roughly $20,000/year) to promote the creation of excellent Forum content.
My question: How would you suggest we run the Prize, or otherwise use the aforementioned budget to support content creation?
(Note that we may wind up sticking to the same format, or something very similar, if we don’t find a different arrangement that we like more.)
If you have thoughts on any of the ideas below, or other suggestions of your own, please let us know on this post! You can also contact me if you’d prefer privacy.
Ideas we’ve heard or thought of so far
- Themed prizes (e.g. setting aside one prize for “the best post on topic X”)
- Giving more prizes, even if they end up being smaller on average
- Selecting at least one judge to represent each major EA cause area
- Including a community vote (not just upvotes, but a separate voting process). This would likely supplement judges' votes, not replace them.
- Having a special “first post” and/or “first comment” prize for people who make a really good first contribution from an account
- Having separate prizes for orgs/professional researchers and people who contribute to the Forum on more of an “amateur” basis
- Having more flexible prize amounts (e.g. maybe one post should win all the money in some months if it’s especially good, or maybe money should be distributed according to vote ratios rather than just first/second/third place)
- Having judges who are somewhat removed the community (or finding some other way to reduce the extent to which the Prize may reflect the biases of the community or of central orgs within the community)
- Getting rid of the Prize entirely without replacing it (one survey respondent, who has written many excellent posts and won at least one Prize, believes it to be “distracting and unnecessarily divisive”)
General comments
In general, I am in favour of having a forum prize of some kind. I personally find it motivating and I think it is a useful way for people to gain credibility within the EA movement. I think it also helps lend the forum an air of seriousness, which is sometimes good and sometimes bad but does probably raise the quality of the content.
From the "giving individuals credibility" perspective, having a few large prizes is good (a $750 prize is large enough to put on a CV for many people, a $50 one is clearly not). From the "motivating good content" perspective, having a moderately larger number of moderately smaller prizes would probably be better.
Perhaps I'm wrong about this, but I've sometimes felt like comment prizes are used to make up the difference on post prizes ("your post was very good but not top-3 this month, so we found a decent comment you made on that post and gave it to that!"). If so I think that is good evidence that we want more smaller prizes.
Comments on specific proposals
This seems like something you might want to do less frequently (annually?) in addition to more regular general prizes.
On the whole I think this would be a good idea (see above).
I don't feel like any major cause area has been getting shafted here; I can readily recall winners on global health, animal welfare, long-term future, and meta topics. In the absence of a specific problem this seems like not a good idea; I feel like it would not provide much benefit and might encourage tribalism/factionalism.
The above comments assume topic-general prizes like the current system; I think themed prizes probably would want some specialist judges involved.
A lot of award ceremonies have some kind of "people's choice" award. If you do this I again think it should probably be less frequent than monthly; perhaps quarterly or annually.
Meh. This sounds cute but I don't actually think it would be very valuable, and could actually be harmful on net. I think it's good to set a high quality bar and incentivise people to work up to it, rather than agonising over making their first post perfect even more than they probably already do.
So I suppose the argument for this is that some people basically get paid to write on the Forum, and it's not surprising that those people (coughSauliuscough) win lots of prizes, which crowds out good content from amateurs. I'm not sure to what extent this is true, and if true, I'm unconvinced that it's bad.
Firstly, a lot of that content is just very good and I want there to be strong incentives to get that stuff on the forum. Secondly, a lot of prize-winning posts are from people who work for EA Orgs but are writing in their private capacity; indeed, you'd expect people who are especially committed to thinking about EA topics to both be more likely to win Forum prizes and be more likely to work for an EA Org. This seems like a good thing we want more of, no less so than contributions from non-Org-employees, so I don't think they should all be put in a special category and forced to compete with each other.
There's a similar argument which is something like "separate out content people write for work vs for pleasure". I think this is more defensible but still probably wrong. If prizes incentivise orgs to post more of their work on the EA Forum, this seems like a good thing that benefits everybody. But I wouldn't be extremely surprised to be argued out of this.
This sounds bad to me, but for fairly vague reasons. I feel like it gives the judges too many degrees of freedom, and that it's probably good practice to have prize amounts be fairly predictable. But I'm not sure about this.
This sounds like a great way to remove most of the value of the prize in return for highly dubious gains.
If I look at the winners of the EA Forum Prize, I expect to see exemplars of great EA content. The best people to judge great EA content are EAs, i.e. community members. We could bring in external expects from adjacent fields (global health, AI, etc), but why?
Has there been any evidence of the prize being divisive, i.e. actually causing conflict? Perhaps there is, but I'm not aware of it. And calling it distracting is confusing to me; is the claim that it incentivises people to write content based on a Keynesian beauty contest, rather than what they actually think it would be best for them to write?
Anyway, I don't have much sympathy with that claim as stated, or for abolishing the prize entirely on that basis, but there might be an alternative interpretation that I'd be more sympathetic to.
I went and checked and I think the people who have won the most money from the EA Forum Prize are actually Buck Shlegeris and Ben Kuhn (tied for first place). I don't know if they were paid to write what they wrote but my guess is they were not.