"I will not have you speak ill of me in my temple! Begone!" he says.

At this point, the commotion has attracted various other acolytes and a few of the guards of the temple ward. The guards seem unsure, but you're pretty certain that they'd side with the deacon over a stranger if it comes to a conflict.