I added this line to the spell and it became much more tolerable to my games.

"The Spirit will only activate in the throes of combat."