The green part is really a general insult, as suggested; however 'черт с ними' ( to hell with them; ~they are gone and forget about them) doesn't fit in this context.