It depends.

If you describe somone as such, you may not offend the person to whom you are talking.

However, if the person about whom you were talking heard you, or if you addressed them...