The thing is, there's a difference between tolerating someone/a group of people and liking them. No-one says you have to like gays, blacks, women, children, Jews, traffic wardens, people with nose-hair, etc., but being able to tolerate them even if you don't like them is the sign of a civilised society. If a society cannot tolerate a certain minority in its midst, then that can lead to discrimination, and ultimately horrific things like "ethnic cleansing".
That said, I would certainly be prepared to fight for your right to exercise your freedom of speech and talk at great length on why you hate Pakistanis so much, or why you think homosexuality is a sin against creation (not saying you hold either of these views, just giving examples). At the same time though, care has to be taken not to allow people with hateful views too much influence, as people in general are stupid and can sometimes be led like sheep by a powerful orator (see: Hitler).
Basically, I do see tolerance in general as a good thing, but I certainly don't think it is universally good. For example, we shouldn't tolerate paedophiles, or "honour killings", or brainwashing. We should, however, try to tolerate people or groups as long as they aren't infringing on anyone else's rights. But it's true that it can be very difficult to know where to draw the line between "exercising my own rights" and "infringing on another person's rights".
I think we handle immigration reasonably well, but if you ask a lot of people over here (natives I mean), they will tell you that apparently we have a massive immigration problem. For example, if you believe what the Daily Mail has to say, we're all about to disappear under a huge tide of immigrants. It's just sensationalism, I think. Yes, our immigration system is far from perfect, but it's not nearly as bad as some people think.