That's entirely reasonable, but there's still nothing preventing you from writing down wu jen or bard spells. Those exist in the spellbooks of other chameleons, on scrolls, etc. just like anything else.