How do you manage weight loss aspirations and fueling appropriately?

A small part of the reason I ride is to help with weight loss (though I am somewhat of a believer in the mantra that to a certain point exercise calories burned simply don't matter). I did a two hour ride today and ate a small breakfast and then did 400 cals of tailwind. It's a bummer though that before even really starting the day, I'm already a 1/3 through my daily calories. Anyone else wrestle with this?