Personally, I disagree with you, but only because of game balance reasons. If you find a magic Dwarven Craghammer in a dungeon and you're normally a human axe guy, I think its lame to be punished for using the new weapon. Especially because for the most part, if your Invoker finds a Divination scroll in the same loot, he'll be able to use it with virtually no penalties. (maybe the Dc is slightly lower)