Hi ((Cheryl)). I don't know him well enough to judge, but knowing me...I personally would have been upset. But I know that my daughter and son-in-law occasionally do that. But then they have a little almost 3 year old to deal with getting in and out of the car seat.
And knowing myself...I'd probably be on the computer 'spying' on his activities, LOL.
But I think it sounds like he loves you very much and you know him better than we do. Could you sit and have a calm conversation with him about it, rather than get angry?
I'm so sorry you feel out of the loop.