If you use a perfective verb, you use the Мы form. If it is an imperfective, you use the infinitive.
EM is the first person (ya) form of the imperfective verb.
Remember Let's in English is short...