I share some of the other reasons put forth by other answers. But the best single factor I can use to explain the output of my system is the question I often ask myself: "Is this helpful?"
("Helpful" = giving people information they can use, improving the overall epistemic condition of the EA community, providing more utility than they require time to read/understand, making it more likely that good conversations will happen on the Forum in the future, etc.)
Spam is unhelpful. Insults are unhelpful. Sarcasm and snark are usually unhelpful and almost never actually helpful, with rare exceptions where they help someone get an important point across. Low-effort posts and comments that don't add substantive information are neither helpful nor unhelpful, so I generally don't vote on them.
Whether I disagree with an opinion or not has no bearing on whether it is helpful; I have to pay attention to other features of the opinion, like how well it uses sources to back up its claims or how clearly it is expressed.