I'm writing this on behalf of the mod team. They've reviewed and commented on this post, but mistakes are mine.
We want and value criticism on the EA Forum. EA organisations often make their decisions transparent to the Forum audience in a way which makes good criticism possible. But transparency has its challenges, one of which is that not all criticism is good.
By ‘good’ criticism, I mean criticism which is valuable to spend time engaging with. Criticism which can improve your project, or improve the case you make for your project. I also mean criticism which is good for the world but not for the criticised person or organisation. For example, perhaps an approach to solving a problem, or a the work of a particular organisation, is getting too much funding or attention relevant to its merits. Then, the recipient of the criticism may protest — but the criticism is good.
Bad criticism, by contrast, is criticism which is time-consuming to respond to and complicatedly wrong. Criticism which might still tarnish a brand, but which doesn’t give readers a more accurate picture of the world, or help the target of the criticism to improve.
Part of running an epistemically healthy discussion space is accepting a certain amount of bad criticism along with the good. The ideal wouldn’t be a space with no bad criticism — interventions that ensure no bad criticism would doubtless also reduce the amount of good criticism.
This is partially why the moderation team does not hide criticism that we think is bad. However, we encourage all Forum users to share their opinions on which criticisms are valuable by voting and commenting. As users, moderators can also do these things.
This doesn’t mean that criticism is never moderated. Some of the accounts, comments or posts that are hidden or lead to banning are also critical. However, they are hidden or banned because they break strong forum norms, generally involving hostility, rudeness, or off-topic content, rather than because they are critical.
Practices we’d like to encourage
This doesn’t mean the moderation team takes no responsibility for bad criticism on the Forum. We can promote and assist the development of practices around criticism, without resorting to drastic action like banning. Some practices we’d like to promote are:
Reach out to people before posting criticism of their work (in almost all cases).
- Lizka previously wrote a post about why, how and when to share a critique with the subject of your criticism. I highly recommend reading that post — she also includes a helpful guide with template emails for critics.
- The most compelling reason to do this is to ensure that you, as a critic, are interpreting the subject of your criticism correctly. It isn’t in anyone’s interest to make complicated, false criticism—it’s bad for the person or organisation being criticised and bad for the critic.
- The best reason not to involve the criticised person or org is if doing so would in practice stop you from posting your criticism. You can always ask the mod team for support if this is the case. Simply email [email protected] with a link to your post, and we’ll coordinate with the subject of the criticism to get a response.
- NB- This practice isn’t usually necessary for critical comments. Reaching out for a reply to a comment before posting would mean it is posted far too late to be seen by readers of the post. In this post, I’m thinking about critical posts.
Where possible, give an organisation or individual sufficient notice that they can respond along with your post, i.e. something like a journalistic “right of reply”.
- There is a practice (and in Brazil, a constitutional right) in journalism known as the ‘right of reply’. It’s the idea that the person being critiqued should have the opportunity to respond at the same time the critique is published. This can be particularly important because, as rational as we might aspire to be, seeing criticism can update our opinion without causing us to look out for later rebuttals.
- This is important on the Forum, where you can easily read a post critiquing something, and then never see a comment under that post rebutting the criticism (unless you have subscribed to the post or commented on a thread yourself).
- In a Forum post, this could look like:
- Sending the post to the criticised person or organisation to be commented on, and then sharing those comments as footnotes.
- Letting the criticised person or org pre-write their comment, and know when you plan to post, so that they can post their comment along with the post.
I recognise that these practices may be time-consuming for the critic to actualise, and I don’t want the result to be substantially less good criticism.
We aim to alleviate the burden of this process for both critics and those being critiqued, wherever possible. If we can assist in coordinating this, please feel free to reach out to [email protected]. We could help by (for example):
- Coordinating with the subject of the criticism on your behalf.
- Collecting comments from the organisation on a google doc of your post, and then publishing the critique, with comments as a dialogue (via your account).
To emphasise, we will not take strong moderation action (banning users or moving posts to drafts, for example) if a user doesn’t break strong forum norms, and this post establishes no new strong norms. This post is a statement of our hopes for good criticism on the Forum and a statement of intention to be more involved and proactive in stewarding it.
Note that I changed some uses of the word "norm" to "practice" to make clearer that in the "practices we'd like to encourage" section of the post, I'm not setting up new Forum norms.
Let me know how you feel about this in the comments below, or on Jason's polls here.
I am somewhat against a norm of reaching out to people before criticizing their work. Dynomight's Arguing Without Warning has IMO the strongest arguments on this topic.
First: Criticism is difficult; requiring more effort on the part of critics makes criticism less likely to happen. OP did acknowledge this with
But realistically, there is a social penalty to saying "I would not have posted this criticism if I'd been required to reach out to orgs first." It makes you look lazy. A norm of "reach out to people, unless it would stop you from posting your criticism" is not a viable norm because critics who would've been stopped from posting criticism are unable to defend themselves. In some cases, I expects this norm to stop criticisms from getting written. So I think a better norm is "you don't have to reach out to people".
Second: A big reason to reach out to people is to resolve misunderstandings. But it's even better to resolve misunderstandings in public, after publishing the criticism. Readers may have the same misunderstandings, and writing a public back-and-forth is better for readers.
(Dynomight's post also gives a couple other arguments that I don't think are quite as important.)
My preferred norms are:
That's what I did for my recent critical review of one of Social Change Lab's reports.
There are some circumstances where it makes sense to reach out to people before publishing, but I don't think that should be the norm, and I don't think we should have any expectation that critics do it.
For some reason I find this title delightful. I kind of wish I could have an "argues without warning" flair or something.
I agree with arguments you present above and your conclusion about preferred norms. That said, I think people might have in mind certain types of cases that might justify the need for reaching out beyond the case of general "criticism". For example, imagine something like this:
Now, my view is, even if this is what happens, this is still a positive outcome, because, like you say:
Transparency has costs, but I think they are usually internal costs to the org, while transparency also has external benefits, and thus would be expected to be systematically under-supplied by orgs.
At the same time, I think most cases of criticism are realistically more mixed, with the critic making reasonable points but also some mistakes, and the org having some obvious corrections to the criticism but also some places where the back-and-forth is very enlightening. Requiring people to reach out I think risks losing a lot of the value that comes from such "debatable" cases for the reasons you mention.
Another set of cases that is worth separating out are allegations of intentional misconduct. I think there are particular reasons why it might make sense to have a higher expectation for critics to reach out to an org if they are accusing that org of intentional misconduct. I think this may also vary by whether the critic personally observed misconduct, in which case I think issues like a risk of retaliation or extreme difficulty for the critic may weigh in favor of not expecting the critic to reach out.
One of the challenges here is defining what "criticism" is for purposes of the proposed expectation. Although the definition can be somewhat murky at the margin, I think the intent here is to address posts that are more fairly characterized as critical of people or organizations, not those that merely disagree with intellectual work product like an academic article or report.
For what it's worth, I think your review was solidly on the "not a criticism of a person or organization" side of the ledger.
That's consistent with reaching out, I think. My recollection is that people who advocate for the practice have generally affirmed that advance notification is sufficient; the critic need not agree to engage in any pre-publication discourse.
I think this is much messier than suggested here. Consider a situation where charity evaluator org A performs and publicly publishes a cost-effective analysis of org B. A critic publishes a re-analysis that suggests the cost-effectiveness is much lower than the original analysis, perhaps far below the expected funding bar. Org A may feel the criticism goes to its competence as a charity evaluator, and org B may consider this an existential threat that could result in loss of funding, yet I think a public cost effectiveness analysis simply has to be considered "intellectual work product".
I don't think this is hypothetical. A while ago there was a case where a critic posted some criticism of a cost-effectiveness analysis of a mental health charity, I believe the charity being evaluated was called "StrongMinds". I think that case is similar to what I describe above.
This presents a similar problem to the laziness allegations mentioned above except worse, since the critic may face allegations that failure to change their criticism pre-publication demonstrates how the critic is insufficiently "truthseeking".
In fact, to the extent that their is no community accepted safe-harbor for what is expected, I think there is likely to be a death-by-a-thousand-cuts problem. Critics can predictably expect that they will need to litigate their conduct regarding these meta-issues when they publish (even if they actually do a lot of the what is suggested!), likely in a way that moves discussion away from the content of their object-level criticism.
Again, not hypothetical. The Nonlinear situation goes to this, I think.
FWIW I do not expect people to run cost-effectiveness analyses by orgs before publishing them.