Main thing that sticks out at me is that removing a race's attribute penalty seems much better then those other minors
Nah, cause you always take a race with a penalty to a dump stat. -2 Charisma is irrelevant for the Wizard as much as -2 int is irrelevant to the fighter. You'll always be able to find at least 1 irrelevant stat.