so it doesn't matter who tells, what determine the gender of the expression is the gender of the person we are addressing. if i am addressing a male мой хороший, if female моя хорошая
That's right. Adjectives change gender to fit an object (or, in this case, a person) they describe.