If you really want to "weaken" casters just make counterspelling easier. For example, make it an immediate action and use any spell of the appropriate level instead of requiring a readied action and a specific spell. You end up with two results, first the casters often use up their immediate / swift actions and so cast one less spell per round at higher levels. Second, the worst of the offensive spells will be countered by the opposing caster.

As a side benefit, you end up with most powerful spells not working in battle similar to many fictional sword & sorcery novels.