Count me in the club! Around the house, I wear cotton pj pants and t-shirts (men's white t-shirts that you buy in pack of 5). If I have to go anywhere it sometimes can be exhausting and frustrating getting dressed. On those bad sensory days, my bed can look like a big heap of laundry as I have tried on all sorts of pants, shirts, socks, etc. You get the picture.
I used to wear pj shorts and tank tops to bed. I can't do that anymore. The feeling of the skin to skin contact on my legs drives me crazy and the touch of the sheets on my arms does the same. Now I wear the cotton (smooth - can't have any texture to it) pj pants and the men's t-shirts to bed.
And bras . . .don't get me started. With the constant hug, which acts up to varying degrees everyday, I hate wearing them but need to as I am getting older and things are going south.

Several months ago I bought new bras that were comfortable at the time. Recently, they are anything but. Target had some Hanes ones on sale for $2.50! I tried them on and I'll be danged if they are the most comfortable bras I have ever wore. I bought 2 of them. Hey for $5.00 you can't go wrong. I cleaned out my closet and dresser the other day and seriously I had over 10 bras in my drawer.

All because they feel better at different times. Heck, I have more bras than pants/jeans. LOL
I am the same way with shoes. Nothing confining or tight. That basically means flip flops all the time.
B2Y - You are so not alone on this one and I agree with whoever stated it was neurological. It is a sensory thing. When I filled out my SSDI paperwork I put in there that there are many times that all of sudden my clothes are so irritating that I need to take them off, immediately.