I think my take would vary depending on the level of weird.
For slightly weird stuff, as discussed in Tessa's answer, it's not actually obvious this is a net cost – the gain in memorability and inferred bravery/independent-mindedness could easily outweigh any costs to credibility.
For very weird stuff, I think it probably is net bad for your expected impact / career prospects, so if you don't care overmuch I'd advise against it. But if conforming to conventional appearance norms is very difficult/alienating/demoralising to you, then you should probably do the weird thing. In that respect, I put it in the same category as having kids, or living in a non-standard location with worse network effects. To quote Julia Wise:
Some things I can do cheerfully. It turns out that giving up children was not one of them. Other people would have no problem giving up parenthood, but I suspect that everyone has something that would cause an inordinate amount of pain to sacrifice.
So test your boundaries, and see what changes you can make that will help others without costing you too dearly. But when you find something is making you bitter, stop. Effective altruism is not about driving yourself to a breakdown. We don't need people making sacrifices that leave them drained and miserable. We need people who can walk cheerfully over the world, or at least do their damnedest.
Now, all that said, what counts as "slightly weird" vs "very weird" will vary depending on the place you live and the sector you work in, and so needs to be evaluated on a case-by-case basis. Academia values conformity in appearance far less than, say, the US Government. And of course different kinds of "weird" will imply different things – kooky vs messy vs uneducated, et cetera.
It's also sadly probably true that if your appearance is unavoidably different from most of the people around you, that will impact how additional voluntary weirdness is perceived – though I'm very unsure how that last thing cashes out in practice, in terms of the impact of marginal weirdness on your reputation.