does he drink also -but not have a "problem" with it?

blackouts and such are bad things

my bro has those once in awhile...
he goes way beyond his limit those times.

I think some just can not tolerate drinking, whether it is a little or a lot.

You need to do what is best for your health and well being, it's sad that your dh doesn't understand this..

maybe he doesn't like the changes or makes him feel guilty

It is your life do what you need to live it the best you can.