I can relate. I have a lot of difficulty concentrating on conversations, particularly if I am distracted by doing something else, or if my husband doesn't mute the TV or look at me when speaking. I too thought it might be hearing (as did my husband).
While I am sure there is probably some hearing loss, I am convinced it is the old jello-brain clogging the information highway between ear and brain. Sometimes its just trying to follow when there is noise or I'm tired. Sometimes its the spaciness of cog-fog.
(Short story

There was a day last summer (pre dx) when my husband and I were having a conversation. I said something, and it took a moment for him to respond. I looked at him and had no idea what he was talking about and said so. He explained he just answered my question. In that moment between my question and his response all memory of our conversation had disappeared from my head. That was pretty scarey and weird (and yet at the time I dismissed it).