Сегодня я хотел бы взять [с собой] некоторых клиентов на ужин.
Sounds a bit awkward, but grammatically correct. I would use несколько клиентов, or кое-каких (if you know who they are).

"Забрать" implies that you take someone at the specific place and after that you go with them somewhere.
Я заберу детей из школы, и мы вместе поедем в кино.
Я заберу клиентов в офисе, и мы пойдем на обед.

In this context "взять" means these customers are invited by you. You may come there together or separately, it's not specified.