For big fights with lots of mooks it's more interesting to take the average dmg because it speeds things up.
For a fight against the bbeg or big scary solo creature its better to roll, it adds some tention. Also if you want to fudge some numbers to save someone from dying straight out, its more interesting to roll.
The encounters with the lots of mooks is mostly just to get the andventurers to burn some resources, so the dmg there isn't really the main concern.

That is how our DM sees it and I do agree.