We have rules. And implementing "situational limits" is also known as "designing an encounter."
I'm not "inflating" anything - I'm using the printed Improvising Damage value from the table in the DMG, which you're supposed to use for things like jumping into a volcano vent.