I found the discussion between mishau_ and Юрка about whether a non-believer should креститься ("make the Sign of the Cross on oneself") quite interesting.
I was raised Catholic, and in Roman Catholic teaching, only baptized Catholics are allowed to receive the bread and wine during причащение (Communion). Not all Protestant churches have such a rule -- in some cases, any baptized Christian from any sect can receive the bread and wine.
However, the Catholic argument is that Communion is very unlike the "miracle of the loaves and fishes" -- to which everyone was welcome, including tax collectors and whores -- but is a special таинство (sacrament) only for properly baptized Catholics who have been properly educated about the ritual and have recently made a proper confession.
Now, I'm a non-Christian and do not believe that the Supreme Being would be offended if I were to "receive the Body and Blood" or participate in other Catholic rituals without belief in them. But on the rare occasions when I attend Mass (generally for family reasons), I do not do these things: I don't take the bread and wine, I don't "cross myself," I don't recite the Catholic prayers that I learned as a child, etc. Sometimes I will sing hymns that are based on the Psalms or other parts of the Ветхий Завет (Old Testament), but I usually avoid adding my voice to hymns about Jesus.
But this is not because I fear punishment in the afterlife; it's partly about respecting the beliefs of my Catholic relatives, and also that I want to set a consistent example for my nephew. (He's not nearly old enough to understand that there a variety of religious beliefs and that billions of people don't believe in Jesus -- but someday he will, and I don't want him to say, "Uncle Rob, if you don't believe in Jesus, then why do you make the sign of the cross when you come to church with us ?")
P.S. And incidentally, although I'm not Jewish, I have from time to time visited synagogue services with Jewish friends -- and I made sure to ask ahead of time: "Are there things I should NOT take part in as a non-Jew? Would it be insensitive or deceptive if a non-believing Gentile puts on a kippa/yarmukle?", etc. Again, it's not that I believe God cares much, but I know that Jews may care about such things...