I can't judge whether you're right to be upset or not. IMHO, guys say a lot of stupid things because they don't think and they don't realize that we're going to overanalyze it 7 ways until Sunday.

Why not wait until you're calm and then talk about spending some real together time - a date night or something. Talk to him about how isolated you feel - that you know that he needs to focus on work, but that you'd like to feel a part of the outside world too. One last thing - do you have any outside interests or friends that can get you out of the house? It sounds like you need it and relying on one person for all that can be hard for them - maybe you could start trying to meet people you can get out with.