I don't really follow this, as the word house is never mentioned, or is it just implied. I thought come to me, could mean anywhere, even if you were at a party somewhere else.