I'd be inclined to say no. It seems a little late to retcon for a fairly minor reason. As to if it was unfair that is tricky to say without a number of other factors such as the level of death in the campaign, if the player has run across the concept of circles containing things within them, etc... I don't agree with people saying don't have NPC's use save or die spells, if you are that worried about those then just remove them completely from the game.

Still less embarrassing than when one of my characters had their entire skeleton sucked out through their index finger (the physics of which slightly baffle me).

I'll definitely second that if the tower was empty don't make them waste their time exploring after that, you can skip and handwave if that will work better for the session (which it almost always does).