First, player agency is not an on-off switch. It's a measure that goes from zero to theoretical infinity. So it's totally reasonable to take away some of the player's agency, not all of it. For example, you might consider a Wisdom save or similar to see if some of their spells prepared/known get temporarily changed in favor of those aligned with the Primordial. Or you might add a chance of earthquake to spells which are resisted by opposition.

This said, visions, dreams, portents of doom and other hallucinations go long a way. You could do with just them and sit back and watch what the player goes and does in response.