A lot of people have mentioned knives, but I'd also like to mention clubs and club-like weapons (crowbars, baseball bats, nightsticks, Maglites). At least in a psudo-realistic setting, you probably don't want everyone you encounter dead. The ability to injure/incapacitate a person without the risk of bleeding them out or damaging internal organs (intentionally) is probably worth carrying around a stick of wood.

That, and tear gas/flash-bangs are expensive and far more noticable when used.