They're rude, it's inexcusable and you are absolutely right in calling them on it, but it can wear you down. I have a slight hearing problem and, often, I tell them to face me when they speak--it's annoying and they think I'm being a PIA patient. We've got to make them realize what they're doing and, if it's uncomfortable, so be it.