идти is about process itself, пойти is more "abstract"

compare: "Пойдём в кино?" (let's go to the cinema?) - here you don't want to mind every your step and without context even cannot answer the question "when?", you just propose to get to the movie theater sometime, somehow.

with "must go" you should, however, use идти: "Я должен идти" (now)
"Я должен пойти" is also possible, but you should add Where: "Я должен пойти к Робертсонам на свадьбу!" (may be, in couple of months)