that's one way to look at it... the other is that if you look hard enough you'll find a racist thought or behaviour in anyone..or at least something that can be construed as such..
I prefer making a difference between .. say.. being mindful of the risks involved in walking at night through a dark alley and finding yourself the subject of attention of some illegal immigrant who's got no business lurking in the alley to begin with, and looking down your nose at someone just for being an illegal immigrant, just because he's ahead of you in the checkout queue at the local mall.
the first situation could be construed as racism too, but let's face it.. anybody you find "lurking at you" would scare you.. not just an illegal immigrant... and if you add to that the fact that statistically (at least in my neck of the woods) some shabby looking illegal immigrant roaming the streets late at night is more likely to be a source of trouble than your average local guy who is too drunk to find the way home, then a person has some justified cause of concern in that respect and shouldn't be labelled a racist just because s/he checks his wallet/purse.