Compromise then?

Rather than having multiple small modifiers, using a single initiative reroll that's treated as having something advantage/disadvantage, using your original initiative as one of the rolls.

For example, the Rogue smooth talks a barmaid into giving him a peck on the cheek after helping him out with some information. This gives him a "positive" boost to his initiative, so he just rolls his default initiative again. If it's higher than his original initiative roll, he keeps the new roll, otherwise it stays the same.

This way, Initiative is modified based off of the last thing that happened, has a natural slope as to how high/low it can go, and cuts down on micromanaged numbers. With how naturally swingy it can be (Initiative of 19, dropped to 7 after a recent negative reroll), you won't have high initiative characters always stealing the spotlight.

I think this would cover most of the concerns people have mentioned (too many rolls, "bean counting", high-low initiative becoming habit, initiative becoming stagnant, maintaining fluid Initiative out of Rests).