I agree.
The X-Men as a metaphor becomes more strained the more other superpowers there are.
Like, a Mutant could deal a considerable amount of damage if they wanted to, so people fear all mutants, and you can explore that Fear.
However, Tony Stark, Reed Richards, Hank Pym, and Victor Von Doom have just as much, if not MORE potential to do harm than those mutants.
It seems like a bizzare dichotomy. Did you get your powers because of a genetic mutation? Hated and Feared. Did you get your powers through exposure to an exploding Alien Wish Machine, everybody loves you.