Are wizards a noticable part of the population of a fantasy world? If they are not, then there likely will not be enough powerful wizards that can be a one man army even if they could do that.

Also, a border is really hard to cover if you are just one person. You may be able to kill a lot of people, but even if they cannot kill you, a large army can bypass you.