A burden? That is so unfair and its not true. We all have different things to give in relationships.
And why would he tell you that? Unless it was to explain why you wouldn't be seeing those people any time soon, then I can't see what is to be gained by telling you.
I'm sorry you are being treated that way.