Another thing which bothered me about the Crimson Mantle was that the paladins did not keep hold of it when they killed the previous Bearer.
I mean, they could easily check that it was magical and it would be prudent to keep it away from the rest of the goblins -- even if only for the purpose of preventing goblins from using a magic item.
It is not generally known by non-goblins that the physical cloak itself is the source of the power. To the paladins, the title "Bearer of the Crimson Mantle" just means that the guy who wants to destroy reality gets to wear a red cloak. Since they don't know that the cloak is what they are actually seeking, they have never actually "gotten hold of it" at all, not in any meaningful way. It has always been passed down just before being captured. It's only existed for about 60 years, remember, and Redcloak has owned it for half of that. So it's only been passed along 3-4 times. In all of those cases, either the goblins managed to swipe it off of the corpse of the old Bearer before the paladins noticed, or they stole it back before the paladins could identify its significance.