Might sound crazy, but I mix up peppermint extract and water and soak a few washcloths in it and toss them in the fridge. If I'm suddenly feeling hot they work great for cooling down and the menthol from the peppermint is quite soothing. And you can rewet them several times with just water before you need to worry about adding peppermint again.

I also have a few mini backpacks (the little purse like ones) that I toss ice packs in if I'm heading outside. Just slip it on and the back stays nice and cool. (Also the clay ice packs seem to work better and last longer than the gel ones)

In an emergency, I always try to keep a few of the 'keep cool' patches in my purse. They're sold with the cold/flu supplies. But they're little one time gel stick on patches. You're suppose to stick them on your forehead for a fever, but I find they work great at cooling you down if you cut them in half and stick them to pulse points like the sides of the neck or inside of the wrists.

I also use and collect the little paper fans, they have really cool mini ones that are about the size of Chapstick folded up, at most party supply shops. I have tons of those in the cars, in my purse and backups in the house. Sure you have to fan yourself with them, but it beats running out of batteries for the mini electric fans that are all but useless anyways.

And a closet full of lightweight linen.