they reap what they sow, but here is the thing, it takes time to raise a militia. when they get stomped, it takes time to send out requests for adventurers. when they get stomped out, it takes even MORE time (and money) to get even more adventurers. In essence, your characters could spend months without being attacked because things take time.
The party could also hire scouts to let them know when danger comes (they can prepare to fight or to hide). They could be the subject of do-gooder adventurers trying to earn their stripes.

They chose to be evil, and that has consequences