This happens to me, too. I notice it more than others around me, except for my son. My son picks up on it VERY quickly and will zero in on the least little word I say wrong.
Part of mine might be due to the Topamax -- difficulty with word finding is a known side effect -- but I did notice trouble with it before I started taking that too. As an English teacher, I'm used to having a much better command of my vocabulary, so it's especially frustrating!
Neurolinguistically speaking, when you TRY to think of a word, there is something called "spreading inhibition" that often occurs that will prevent you from thinking of it at the moment, but it will come to you later when you aren't trying to think of it. This is a phenomenon that happens to everyone, and it's probably compounded by our extra neurological issues. So, probably the best thing to do, as Mark suggests, is not to make a big deal of it as much as possible and expect that it will come right sooner or later.
With friends and family, I have told them all that Topamax has this side effect, so they know to roll with the crazy.