Roling dice takes time (and counting the result too).
The PCs usually like to roll dices, so it's not wasted time.
The PCs usually don't care if the monster roll dice, so it's wasted time.

Few exceptions:
One PC is near 0 HP -> Rolling damage dice against him add tension, and is fun.
The attack is something important (first breath of the dragon, high level spell, ...) -> Taking the time to roll the dice show that this attack is important.

Mixed version:
Rolling only part of the dice -> DON'T do this. It has the disadvantage of both rolling and not rolling.
Rolling only the dice of few enemies (either always the same, or arbitrarily depending on your mood) -> works pretty well.

Do you need to change the CR? No, randomness in the attack roll are far more relevant than randomness in damage rolls.
Maybe an exception for the first few levels.

Does it get better or worse? It get better at high level. 8d6 will rarely makes any result different from ~40, while 1d6 can do anything from 1 to 6. Moreover, low level characters are limited in what they can do (no powerful spells) to compensate for lack of luck.
The more the levels advance, the more 1) You roll too many dice for random to be relevant 2) The PCs have solutions to every problems (it will just cost them A LOT).
The only random thing that get worse are "save or die" spells, but those spells are save rolls, not damage rolls.