No, both sound fine.

Talking over the phone is maybe a bit more "business-like" or formal, and it implies that you spoke about a particular topic, perhaps.
Teenagers might not use "over the phone" when they spoke about something, but adults might.

"Mr Jones and I discussed this matter over the phone yesterday".
"Jim and I talked on the phone yesterday".

You can also say "chatted on the phone" - then it means two people who know each other well, probably female, talked for a while on the phone.