Food affecting baby?

jlb08

DS has always been gassy as most babies are. I tried to cut out milk to see if that made a difference. The nurse told me it wasn't necessary to cut out all dairy; I'm not sure why since milk protein would be in more than just milk. Anyway, it didn't seem to make much of a difference. But yesterday I had ice cream for the first time in a while and this morning DS seems very gassy and miserable. I feel so bad for the little guy.

Is it possible that he's reacting to the ice cream or am I overthinking things? I eat other dairy products, especially cheese, pretty often and that doesn't seem to affect him much. Could it be just ice cream that upsets him? Or maybe it's just a gassy day. He didn't poop at all yesterday, which could be adding to his discomfort.

Everyone is quick to ask about my diet when the topic of LO's gassy episodes comes up ("What are you eating?" "Maybe it's something you're doing."), so I think I'm overly worried/guilty now about what I eat. 

Any thoughts or experiences with this? 

ETA: Also if it is food that affects LO, about how long can I expect it to stay in his/my system? Thanks!