It means that when you see his face you want to take a brick and to throw it to his face.
Depends on yours and his personality and behaviour. People are different.

Looking at Lampada's and Serge_spb's posts you can see how they feel it.