I never payed attention to this, but I think that people usually don't announce that they go to the bathroom, especially in the mixed company or in formal or semi-formal setting (I can't imaging this happening during business dinner with people you barely know). They say something like "Я отойду на минутку" without explaining their intentions. Everyone can fill the gaps anyway.

I'm trying to remember if I ever say "Я иду в туалет".. Hmmm. If I do, most likely I won't say it for everyone to hear, maybe to a friend or someone, who sits nearby in case they'll need to find me.

On the other hand if you are among close friends, it's possible, and no one will think twice about it.