You might want to drop spells altogether. It would be more GM-intensive to decide what they can or cannot do, but the way you describe Magic, it sounds like this would fit.

Instead of spells, I'd say you have a number of attributes that define the limits of what they can do with magic. For example, perhaps they could have a magical Strength score that defines the raw strength their magic can have. It could determine how much weight they could lift with it.

And you know, something like that.