I always heard it explained as an "unjoke" that you would use when a joke was expected. The humor being the unexpectedness of a simple, direct answer in a chain of jokes.