I have to - is a set expression in English.

usually translates into Russian as должен, надо. (close in meaning to 'must').

So 'I have to go' corresponds to Я должен идти. Мне надо идти. (implies I have to go due to circumstances)

У меня - is used when you mean 'possession'. I have a book = У меня есть книга. (Lit., By me (In my possession) is a book).

As for У меня идти. У меня пойти. У меня есть идти/пойти. - they all do not make sense, as there is no possession described here.