I would suggest modifying the wording of Prayer Mastery slightly so that rather than any spell learned from a non-cleric, it only penalizes spells that aren't on the cleric list or have been learned at a lower level than they appear on the cleric list. For example, an archivist who learned Cure Light Wounds from a druid is not gaining any benefit from having learned the spell from a noncleric source, so there's no real reason to penalize him for it.