The grammatical gender of these words depends on the object they refer to. Мой брат = my brother, masculine, моя сестра = my sister, feminine. Note that this is a grammatical and not just a biological distinction, so you also get моя машина = my car. But actual biological gender takes precedence, so you'd say мой папа = my father, even though the word папа ends in -а and is declined like a feminine word.