They're just using "a bit of your soul" as an elaborate way of saying "a bit of XP." The way XP functions in the universe isn't clearly defined, so having it be part of your soul is totally fine, imo.