Start small and build up. Start with a village, town, a neighbourhood in a city. You can name far away places but don't bother with the details unless the PCs actually go there. And don't worry about changing your mind. Tales from far away are just that, tales, and often wrong with only a hint of truth.