What do you think?

Should "whom" be phased out of the english language?
I am sure you can explain how the rules work, but language is not what is codified in some gramatical rules but what people actually speak with each other. And it's not even an issue of non-native speakers, even a big number of native speakers appear to never use it at all.

In German, the genitive article is increasingly disappearing and even though I consider my speaking quite close to the gramatical standard, this is one thing I don't think I ever use at all.
Das Ende vom Genetiv ist mir ziemlich wurscht.

But what do native english speakers think? Is it something you even notice when people don't use whom when it should be used, and it something that makes anyone who does do it appear to speak bad english?
I've looked up one site that has a couple of examples, and intuitively I think I would have used "which" instead of "whom" in most places. Or is that even worse than who?
Or are you among those who could care less?
(Sorry, had to make that joke. ;) )