Sometimes I have to defend myself from strangers who suggest I should be doing/trying to do things I can't do, and I HATE it. It makes me feel trapped and miserable while they yammer on, but I try to be polite...

I'm sorry you had to go through that, BF. It doesn't feel good when we feel like we are being judged, or to be reminded of our limitations

You just lost a lot of weight, and you should be proud of yourself!
To hell with him!
I think it's a good idea to bring it up in your therapy group though. It's a controlled setting, and it would be a good opportunity for you to assert yourself, and perhaps learn how to deal with situations like that alone in the future