You should limit the extra spells known/prepared to level 5. Being able to prepare extra 6-9th level spells is too strong, since it's a deliberately scarce resource.
Maybe at higher levels you can record a 6-9th level spell as a rune (which costs a spell slot), and infuse it with a discounted metamagic ability.