I went for a hearing test yesterday and had a good chat with the audiologist. My hearing is all "normal" on all tests. But he said I have hyperacusis and said that since I've been wearing earplugs for so long it may be making it worse.

He suggested I slowly stop using them in order to desensitize my ears to normal sounds. I will try it out. I had never thought of this but it does make sense.