It’s simple: I consider clothes a form of expression, just like words. They have a utilitarian side, but words do, too. Past that, they are a way we express ourselves. And not just with text on T-shirts. The clothes we wear, from our hats to our shoes (even the clothes no one sees) are ways we communicate to the world. Therefore with extremely rare exceptions, no one should infringe on the choice of what you want to wear. To me the only exceptions should be in the interest of public safety and when I enter into a contractual agreement with someone. Otherwise, I should be able to wear what I want.
This doesn’t mean I should expect everyone to like or accept what I wear. But I should have the right and the ability to wear it, and I should not be legally limited or illegally discriminated against for wearing it.