Hmmmm... I myself have never felt this way, but I think I know why.

Actually "для" should be unstressed, the "я" is pronounced like "и" or like "е" here ("л" is soft). And no, it shouldn't sound different in "для меня" and "для Вас" at all. The letters "м" and "в" don't play a part.
- Оля

Vowel Reduction is a prominent topic in learning Russian and affects (basically) every word.
There are a few rule groups, and the е-ё-о triad comes into play a bit.

This is why Москва is pronounced 'muhskva' not 'mohskva'

It's a bit complicated. If you were to say для in a vacuum with no surrounding words - it would be with an /a/ sound. The thing is, prepositions [,words in general really,] never exist in a vacuum -- it's always going to be said with a word following it, meaning it's always going to be unstressed and the я will always reduce to a more relaxed ~е

I would say, for the time being, think of it as a я, say it as a я, and try to hear it as a я, and the rest will come with time. [mass data exposure]
After a while, the vowel reduction rules can start to feel like a very natural filter to the point where you hardly can tell that they are even happening.