The class does have access to every spell in the game. Any individual character may not (yet to be determined), but the class itself definitely does.
EDIT: Actually... it gets a whole lot more interesting now that I've realized that it's descriptors ONLY, and that subschools aren't included. Now you can't even learn most abjurations at all unless you go for the whole school. Hmm... still an unbalanced class, but already pretty restricted.
They can get a free +1 metamagic on every spell at 1st level for giving up one descriptor. If they give up two descriptors or one school that's free +1 metamagic and +1 DC or caster level for all spells they cast at 1st level. It gets worse from there. You're looking at potentially +2 levels of free metamagic at 2nd level with Specialist Adept. By 20th level a character could have up to +6 levels of free metamagic with their spells before taking feats into account.
I'm going to suggest that you eliminate the concept of Caster Points entirely. They are too fiddly and add far too much potential for abuse. Beyond that I would suggest limiting the choice in variety of spells to ONLY descriptors, and only non-aligned descriptors. That brings the mechanics more in line with the theme you seem to have in mind. You probably also want to account for Subschools, if only to remind players that (Calling), (Creation), (Charm), (Figment), etc are not descriptors.
EDIT: More necessary restrictions: 1) Make a note that Spell Savants learn spells at their highest available spell levels. This prevents abuses such as learning Haste as a 1st level spell. 2) For learning spells with multiple descriptors require the Spell Savant to have chosen all of the descriptors. That way people aren't getting freebie spells that belong to two "disciplines."