Nouns of common gender (общего рода, marked as м. и ж. in dictionaries) can refer to objects of both genders. They are often emotionally charged and speak of some qualities of objects, or refer to their occupations:
староста, шалунишка, умница, неряха, жадина, непоседа, недотёпа, торопыга, etc.

Both verbs and adjectives in this case are changed accordingly and match the gender of the object in question.
Наш новый староста объявил о собрании. (male)
Наша новая староста объявила о собрании. (female)

Words defining occupations ( м. in dictionaries)
Some words defining occupations can also refer to both genders: врач, учитель, менеджер, etc.
In this case verbs should match the actual gender of the person:
Врач уже пришел (male)
Врач уже пришла (female)

But attributes/adjectives are always in masculine:
Он хороший врач
Она хороший врач

That's why such words are considered to be of masculine gender (NOT common gender).

Masculine nouns ending with -а (-я)
Папа, дядя, юноша, юнга, слуга, братишка, etc.

These nouns just happen to look like they are feminine, but they "act" like usual masculine nouns and should not be confused with nouns of common gender.

So... "крошка" belongs to the first category, thus technically it's correct to say "Мой сладкий крошка". But such a phrase is rather awkward, probably because it's something that is more likely to be said to women, and "крошка" with its "feminine" ending does not help.

Please note, that people often disregard these rules, going with what their guts tell them and saying things like "Наш врач пришел" (about a woman) or "моя кроха" about a boy. So don't be surprised. ))

Does the М. и ж denote Muzh and Zhena? As in you can use both?
Мужской и женский (род) - masculine and feminine (gender)