If one person has an advantage than the other- on a societal level- the one that's better off has privilege. Thats all that means. So yes, people who are being treated fairly do have privilege that people being treated unfairly do not. Because privilege is comparative.