Sonny,
You're not going nuts. I'm very sensitive to cold too. When in the car I close off the vents when my wife has the A/C on and the house is kept warmer than usual. Once I sat out in the yard in the sun wrapped in my winter coat and thick jeans freezing and it was 95 degs out!
Sometimes I'm hot, sometimes cold, but never the temp as it really is. It's like my body has a mind of it's own. And the flare ups, grrrrr.
I just wanted to let you know you're not alone...