The whole party is feeding this guys abilities, meaning that if you nerf them you effectively nerf the whole party. My suggestion is to keep a couple beefy but low offense minions around that can tank your tank. Gargantuan or colossal vermin are good for this, or summoned celestial bisons/fiendish apes. They make for good stories (it was huge!), they extend combat and they aren't likely to kill of your party members.