We could say the exact same thing in english... it didn't really ever confuse me... for example.

"Are we going to the fair?"
"No"
"Are you sure? " "
"yes"
"But you said we were going"
"we'll...