I think it's pretty solidly canon that he doesn't know how to fight or defend himself. The question is whether he's so tough that he doesn't need to.