How did you rectify differences in desired family size?

My husband and I agreed before we got married that we wanted two kids. We were both very confident in this decision.

Fast forward several years and we had the two kids (a girl and a boy just by chance, ages 3.5 and nearly 2). I'm enjoying motherhood way more than I thought I would but I also feel like our family is complete and have no desire for another.

My husband has since decided that he wants at least one more. He's not pushy, but he also has trouble seeing our family as complete. As he puts it, he feels like there's at least one empty place at the table.

He's leaving the final decision to me (as is appropriate since I'm the one going through pregnancy/breastfeeding/handling most of the day to day). I'm totally ready to be done and worry that a third child will ruin our wonderful family dynamic, or that the added stress will turn motherhood from something I enjoy into something I frequently don't. I do not look forward to the logistics of a third kid, the mom guilt from not being able to be there as much for any of my kids, etc.

At the same time, it breaks my heart to deny him this. I've never seen him want something so badly in his life. He is a wonderful husband and father. The greater share of parenting falls to me, sure, because my job allows better work/life balance (and biology, at least for the first few years), but he is involved and we are more co-parents than anything. I can tell that another kid would bring him SO much joy.

I just never, ever saw myself as a mom of three. I never wanted a big family. I would have initially been fine not having kids at all (although now that I have them, I'm so glad I do). We also put off having kids because we weren't financially ready (and I wasn't mentally ready) for some time, so I'm now 38 and he is 43. I'm still nursing my youngest, and I know how hormones can affect mindset, so I'm waiting until I've weaned him (within the next few months) before I "finalize" my decision, but I doubt my opinion will change.

So my question is to people who have been in similar boats, where one person really wants more but the other doesn't. How did you decide which person's wants to prioritize? How long has it been since you decided? How do you feel about your decision now?