Look for a change in color on your cheeks, itching/hives, shortness of breath, dizziness, nausea, swelling at the site, etc. Like Wiz said, take a few benadryl just to make sure if you want.
When I was a kid my mother would make us put mud on the sting and let it dry. Just another old wives tale? Some say baking soda paste, some say ice, I don't know what the medical peeps say these days other than use a credit card to remove the stinger and watch for signs and symptoms of allergic reaction.
When I came home from my festival a few weeks ago, we discovered a nest of yellow jackets under the front stairs. No stings, but the next day I got a can of that newfangled foaming wasp and hornet killer and had the old man flip the stairs and the landing and coat them with the foam. They left...