I say shoot them all and take their stuff. I bet if you went with the artifact, they'd make you find one first and then the bandits would shoot you anyway.