European nobolity is pure chaos. And it gets worse when europeans used the same terms for nobles they encountered in foreign societies.

That said, I think most of the time its tradition.
Some people may now cry out in horror, but I think at least for the germanic people the whole system started with the leaders of small communities and warlords. Over time alliances between communities became more permanent and a tradition evolved that the leadership of the group would be passed down to the heir of the old leader. And all over the world, they developed different terms for these leaders.

Also, I terribly simplefy things here!
I would say basically there are three types of noble ranks:
- Hereditary sovereigns.
- Appointed officers.
- Emperors.

Hereditary sovereigns are rulers who own their land and possible even the people as their personal property. Those are for example Dukes and Kings. In the Holy Roman Empire, the title of King mostly indicated that you were really powerful, but it mostly worked like all the other Dukes and Archdukes, which also were just powerful Dukes.
On the other hand you have for example the Counts. A count is an appointed administration officer who manages parts of the realm of a higher noble like a Duke or a King. In many cases the title was passed down from father to son, since the son would also be the Counts apprentice who learned the job and all the local circumstances as he grew up, so he was the most qualified person to do it. But if the sovereign didn't like the job they did, they could be fired and someone else be appointed.
Below that, you also usually had the Knights. As a knight, you were allowed to join the club of nobles, but that didn't provide you with any political power. A knight could become very rich and powerful by managing his small piece of land well, but they could also be just as poor as any peasant.

Emperors are special, as they rule over many nominally sovereign rulers like kings and dukes, and it often also means that they rule over people from many different ethnicities. This could be a King who just was so powerful that he effectively forced the other dukes into line, like in Russia. Or it could be one of the sovereign rulers whom they elected from within their ranks to be the chairman of their allianace, as was the case with the german Emperor.
The "Emperor of Japan" is a particularly strange case, since he almost always ruled over a single people, the Japanese. However, for a very long time, he was mostly symbolic and all the local Daimyo ("Big-Landowners") did whatever the hell they wanted.