(This is a 2016 post that someone recommended I crosspost)

People don't usually volunteer details about why they decided to do something, how they did it, or how it turned out, unless they have another goal in mind. You see people and organizations writing about cases where they've done better than expected, in the hope others will think better of them. You see writing that explains already-public cases of failure, casting it in a more positive light. You see people writing in the hope they'll be seen as an expert, to build up a reputation. Additionally, while most real decisions are made in people's heads as the output of a complicated process no one really understands, if you look at decision writeups you'll typically see something easy to follow that only contains respectable considerations and generally reflects well on the ones publishing it. If you're trying to learn from others, or evaluate them, this isn't much to go on.

Efforts to change this generally go under the banner of "transparency", and this is one of the components of the effective altruism (EA) movement, especially for EA organizations. GiveWell is especially known for putting this into practice but most EA organizations value transparency and prioritize it to some extent. Many individual EAs do this as well; for example, as someone earning to give I keep a donations page and have posted several spending updates.

This puts the members of the EA movement in a position as consumers of transparency: people and organizations are releasing information because it benefits the broader community. This is information that they could easily keep to themselves, since as a practical matter everything is private by default and requires effort to make available. Writing a report requires taking an enormous amount of detail and deciding what to communicate, which means it's very easy through selective inclusion to hide mistakes and present yourself or your organization in an artificially positive light. And not even intentionally! It's very easy to subconsciously shy away from writing things that might make you look bad, or might reflect badly on people you on the whole think highly of.

So imagine an organization makes something public, and voluntarily reports some kind of failure that people outside of the organization wouldn't have known about otherwise. If people react critically and harshly to this failure, it makes them much less likely to be willing to be so transparent in the future. And not just this organization: others will also see that sharing negative information doesn't go well.

When people react negatively to an organization sharing an instance of failure one thing they're doing is putting pressure on the norm that organizations should be sharing this sort of thing. If the norm is very strong, then the pressure is not going to keep people from sharing similar things in the future, and it also means that seeing a failure from this organization but not from others is informative. On the other hand, if the norm is weaker we need to be careful to nourish it, not pushing it harder than it can stand.

29

2 comments, sorted by Click to highlight new comments since: Today at 6:28 AM
New Comment

I think a problem here is when people don't know if someone is being fully honest/transparent/calibrated or using more conventional positive-slanted discourse norms. E.g. a situation where this comes up sometimes is taking and giving references for a job applicant. I think the norm with references is that they should be very positive, and you're supposed to do downward adjustments on the positivity to figure out what's going on (e.g. noticing if someone said someone was "reliable" versus "extremely reliable"). If an EA gives a reference for a job applicant using really transparent and calibrated languages, and then the reference-taker doesn't realize different discourse norms are in use and does their normal downward adjustment, they will end up with a falsely negative picture of the applicant. 

Similarly, I think in a community where some people or orgs are fully transparent and honest, and others are using more conventional pitch-like language, there's a risk of disadvantaging the honest and generally sowing a lot of confusion. 

Also, the more everyone expects everyone else to be super honest and transparent, in some ways, the more benefit to the first defector (since people might be more trusting and not suspect they're being self-promotional). 

I’ve talked to a granter and have the impression that transparency about their granting decisions is similarly tricky, as it’s often particularly sensitive for the receiver of rejections. It’s hopefully common practice to add a disclaimer what a rejection means, that there is unfortunately a big time bottleneck on the side of granters, and that the feedback (if given) is fairly unpolished and that sharing it transparently is a sign of trust that the receiver acknowledges this tradeoff.