True, but the Raiders (hypothetically) kept the loot from the "heroes" separate from everypony else's stuff. If nothing else, he'd know that one of the slaves on the train was a Stable-Dweller, might not know it's one of them unless they bring it up.
What's that? You can't hurt me because you're a construct of our imagination? Darn, stinks to be you then. *imagines Fox Trot with a fanciful tiara*
And besides, inspiration for the Ship Which Shall Not Be Named struck when I was reading about you flirting with Pretty Pink. So clearly, this all starts with you somehow.