Sometimes I hear

говорить по-русский

And at other times

говорить на русском

I asked my Russian friend and this is what she had to say:
"Its a little complicated. They r even the same in idea so I can't even explain the difference
But the second one is more correct"

Maybe you can help me with a more accurate explanation?