If that's how you feel, houserule it away. It's your game, D&D police is not going to your house to arrest you. Boost (or nerf) druids (or any other class) to your (and your players') heart's desire. What I find odd is the insistence that allowing Druids to wear metal armor is not a houserule.
For me, druids are fine sitting around Dex 16 until around levels 6-8, where it's ok to give them a boost. It's just convenient (and probably not even intended by the designers) that the metal armor restriction rule in the PHB allows it to work like that just fine.