Oh, I don't know, Jay. I guess sometimes I just wish I could be with you instead of Tom.
You know I feel the same way, Daisy.
I think this is the conversation that took place because they were finally alone together and they really just wanted to tell each other how they felt. Gatsby knew Daisy wasn't crying over shirts, and he wanted to try to get her to spill the real reason. When she does, he is super content with her response.