This argument comes up with sexy armor. It hardly ever comes up with sexy clothing. The sexy armor is usually still heavy and/or rigid and very certainly uncomfortable. Yes, it is usually far worse then plain clothing arguably worse than being naked.
Sexy clothing however is usually not worse than other clothing (baring climate concerns). In any setting, where people go into battle in normal clothing doing so instead in sexy clothing would not be a problem
Going unarmored into battle when armor is availableand helps against contemporary weapons is also rarely a good idea, regardless how that armor looks. Most people going into battle without armor did so because they didn't have any or it was ineffective against contemporary weapons. The rich barbarian fighting the knight naked ia just an idiot.
It is fast to equip a helmet. You won't have it on all the time. But yes, someone with any significant armor should have a helmet somewhere.