My DH and I used to go out a lot. Usually on the weekend, to a favorite bar/restaurant or the American Legion or the VFW. I started having panic attacks, right in the middle of things and had to GET OUT NOW!!!
I did have a talk with my dear understanding Hubby and it was written that all I had to say was "Lets go" and we would leave.
I went to my Neuro/Psych and he said PAXIL and I said Yes. It saved my life and DH was, then, also happy.

He's gone to Heaven now and Paxil quit working for me, after 6 years. I'm on Prozac now...not as good as Paxil but gets the job done.
Your DH is scared. A Calm talk and a plan for the two of you is in order, as the others have said.
Good luck, Dear One..