Actually, I think 2/3 is right. Elf, Dwarf and Halfling have their own lands, and are definite minorities in Human lands. Also, in many Human societies throughout history, women are denied advantages that men do. Minority isn't the correct word, but disenfranchised group might work.
Roy is possibly not a minority in his own hometown.