Serge you are right. The problem lies in the connection of the vowel and the preceding consonant. But I'd like to add that there's more than what you said. The problem is a bit deeper. Many languages have similar sounds like a Russian B and an English V or B/Б or something. There are also many similar sounds in French and Russian or French and English. Literally every language is similar to others in terms of how they sound. But there are also slight differences in the articulation of seemingly identical sounds between languages, which are normally called accents. These differences come from the shape of the tongue when we speak. These can be inherent to pronunciation of individual sounds, like the tip of the tongue in L touches or doesn't touch the teeth (just for instance).

There are also those going through all the sounds, through all the speech, they influence all sounds. This general feature of the Russian language compared to the English language is that the shape of the tongue of English-speaking people is flat or even slightly concave like a palm shaped to catch a drop of water. At the same time the Russian tongue is concave in the opposite direction, like an umbrella. This influences all the sounds so that the middle of the tongue in Russian works in a different way.

This very raised middle part of the tongue is used to create, for example, soft consonants. It goes up when pronouncing them and touches the roof of the mouth right behind the alveolar ridge as if squeezing a berry inbetween it and the soft palate. The problem is that the English people use quite a different shape and this part of the tongue that normally goes up in Russian doesn't want to go up for an Englishman trying to produce a Russian sound. They may even not know that it SHOULD go up and where exactly it should bend and touch. They use slightly different points of the tongue to contact with the soft palate when pronouncing their consonants. And try to do it the same way for Russian, which is a mistake. No wonder they may suck in pronouncing soft consonants or Ы or something else.

As you said, the sounds in Russian influence each other when run together. But if pronounced with an incorrect form of the tongue they simply cannot be pronounced correctly. Moreover, it's quite inconvenient. Hell, it's a torture, it's more than just inconvenience. One of my friends said that the Russian Ы sounded like a zombie to him and that he couldn't get it pronounced right. And his tongue used to get tired in three minutes of speaking Russian. When we dug deeper into the problem it appeared that he used his habitual shape of the tongue. When he changed it (it definitely needed some training because his tongue muscles weren't used to working this way) and gotten the hang of it he started speaking more confidently and correctly.

Why does it happen with Ы, why does the shape influence the vowel? Because vowels need resonance in the mouth, in the resonating chamber inside the mouth, formed with the soft palate and a specific shape of the tongue. This resonance chamber in Russian and in English is different because of the shape of the tongue so one who wants to get the right sound should learn how to produce the correct resonance. Simply how to articulate in a Russian way.

One of the funny things I learned during my time learning English is that their (American, English, etc) doctors do not need a stick or a spoon when examining the throat of a patient. Because when they say Ah the concave shape of the tongue reveals the throat hole clearly. On the contrary, Russian doctors do need this spoon or a special flat stick made of stainless steel to push the tongue down when examining the throat of a Russian patient. Just because this raised middle stands in the view and they can't see what's happening behind it. So my advice would be: "learn the correct shape". Take a mirror, open the mouth and learn how to bend the tongue so that its middle hides the most part of the throat hole but you still can see the lower molars. Then hold the tongue in this position for a while so the muscles will learn this new shape. Then learn to give the tongue this position with the mouth closed. Keep it for a couple of minutes. Learn the transition from how your tongue usually lies in the mouth to how it should lie in the mouth when speaking Russian. And finally, use this middle of the tongue when producing Russian sounds.

Kinda like that.