I suggest you try to reduces the impact forces when you can and just accept them as part of life when you can't. When I experience a single bump, I just try to ignore it. When the road gets rough, I lean forward and try to reduce the impact forces.

I think your doctor is right. Your anxiety is making them worse than they are.

So, take reasonable precautions when you can and try to accept them when you are caught off-guard.

My best to you.