I'll second Mark's thought on ear wax. It was the first thing I thought of when I read your post. I have squirrelly ear canals, I've been told, and this is something that is a recurring issue for me.

It might be good to have a doctor look in your ear to see if what the issue is... might give you peace of mind and help resolve it more effectively.