It can be specified in the context
For example
я говорил с моим коллегой - I have talked to my (male) co-worker
я говорил с моей коллегой - I have talked to my (female) co-worker