But seriously at that point it would be treated as armor. The shield is treated differently because it's being wielded in one hand and actively used, while the armor is worn.
Shields do come off feeling a little weaker through this, but their benefit should be more in other areas (through skilled uses via feats and maneuvers) than through passive bonuses. Speaking of I should double check with my combat style feats and see if there's stuff there that breaks the RNG or otherwise doesn't play nice with this. Because knowing me there is.