I say that you CAN dislike conjuration and specialize in it without bad RP.
Do you choose things just because you like them? Some people choose things they dislike, cause it gives more benefit.
Does everyone take up their dream job? Or do most of them pick up a practical, well earning job, rather than something they like, but isn't much use?