Will's Alternative Title: Beware Fully General Counterarguments
A fully general counterargument is an argument which can be used to discount any conclusion the arguer does not like. Being in possession of such an argument leads to irrationality because it allows the arguer to avoid updating their beliefs in the light of new evidence.
Knowledge of cognitive biases can itself allow someone to form fully general counterarguments ("you're just saying that because you're exhibiting X bias").
Knowledge of cognitive biases is probably the main way to fall foul of fully general counterarguments, but it's not the only way. Below are some non-bias-related fully general counterarguments that have taken place in my head over the past ~6 months. It was pretty painful writing most of these out, but my hope is that this exercise helps others identify fully general counterargument-themed errors in their own thinking.
"This person is trying to be a counterweight to what they perceive as my extreme view, in order to get me to update by a bigger amount. Their true position is only a fraction as far opposed to mine as they're saying."
"This person doesn't understand what they're talking about. In fact, I'll test out my hypothesis here by quizzing them on a couple details which are kind-of-but-not-really related to the main topic of our discussion. Yup, they don't understand these details to the same level as me, so I can write off the rest of what they've said."
"This person is not a proper longtermist, so they're mixing in values (i.e., what they care about) with their probabilities (i.e., their view about how the world is) in an attempt to convince me that my probabilities are wrong. Concrete example: this person is exaggerating the chance humanity doesn't recover from collapse because they care more about the human suffering during collapse than I do."
"This person doesn't have any EA/x-risk research experience. Maybe I'll listen to them if they go away and acquire said experience."
"This person is more junior than I am, so they're spent less time developing inside views. I'll assign them trivially small weight in my internal parliament / all-things-considered view."
"This person spends all their time community building and not much time learning or thinking. They don't have good takes."
"This person didn't go to a top uni, like I did, and didn't graduate top of their class, like I did. Therefore, I should expect that where we disagree, I'm right and they're wrong."
"I have an IQ of 183, so, statistically speaking, this person is dumber than I am. Probably a lot dumber. Thus, I'll start with the prior that I have little to learn from them."
Will's Alternative Framing: Don't Let Your Intelligence Defeat Itself
Intelligence, in order to be useful, must be used for something other than defeating itself. Knowledge of human heuristics and biases may cause people to selectively find those biases in arguments they disagree with. This ability to destroy arguments that aren't already accepted will make a more knowledgeable person less able to change their views when presented with evidence.