The highest I've ever seen for Rome was 60 million people living within their borders, only about 10 million of them living in Italy. Officially, the Roman Empire seems to have had about 450,000 troops. Given this was at a period of relative peace though, it seems likely that if an adversary arose at that time, the Romans could have raised more.

I'm also trying to figure what the percentage of that population was slaves or conquered peoples that could not qualify for the army. I'm not exactly certain whether or not they could have raised more if those slaves were free or if having slaves required more military presence by such a huge margin that it levels out or favours the army. Given how many slaves were in the city of Rome itself which wasn't defended by the legions directly it seems likely that full time military were not used to counter slave populations.

By percent though, the Roman Empire had fewer men in the legions than they had during the height of the Republic. At the height of Roman military power in the Republic they managed over 700,000 legionaries with a population less than half that of the Empire. That was after the Marian reforms where a legionary was a professional soldier as compared to the Hastati civil soldier.
450,000 legionaries was actually about peak strength for post-republican armies--the empire was always at war somewhere. That's maybe .7% of the population.

The Ming army, at its height was ~1 million, but 70 to 80% of its soldiers spent the vast majority of their time farming. If we put the active Ming army at 250,000, that gives us around .33% of the population. If their entire reserve army was pulled up, you'd crack 1 percent of the total population.

When the Mongol-founded Yuan dynasty wanted to invade Japan a second time, the Japanese mustered about 40,000 troops to oppose them, out of around 6.2 million people. That's around .65% of the population, which is in keeping with the trend we're seeing in the estimates of Roman and Ming troop strengths.

I'm sure there were reserves, private armed guards or retainers, and local militias, but 1% of the population under arms is just extremely uncommon prior to industrial-era warfare, but even in that context only immense wars like WWII actuall produce different numbers. If you look at the size of the US armed forces right now, active, guard and reserve, you're still looking at about .7% of the population.

Damn, this puts a spanner in my campaign works, if a town with 15,000 people in it is threatened by war by a nation with a 20,000 army you'd think they'd surrender right away.
Damn, I need to figure some things out then.
Mmm, not necessarily. There are a lot of other considerations. Remember, just because you have a total troop strength of 20,000, that doesn't mean you can actually employ them all in one place. If that state has opportunistic, powerful neighbors, or a noble class that isn't very reliable, it'd probably be pretty stupid to pull up every active soldier in order to conquer one city state. Moreover, they might not even be able to if they wanted to--mustering a feudal army was not an easy process, and you could generally only keep it under arms for a limited campaign.

A city-state is also unlikely to exist purely as a city--it'd also include a decent swathe of surrounding territory to draw defenders from. An independent city would also be a mercantile center of some kind, with more ready cash than its more agrarian opponent, making it much better at calling up mercenaries.

Historically, city states often produced higher-quality citizen militias than the imperial/royal levies they faced down, and combined with mercenaries, temporary alliances with similar city states, and the logistical advantage of fighting on your own turf regularly fended off opponents who were quite a bit more powerful on paper.