Rice, corn, potatoes, and sugar are all legal. You just have to use different flour. It's available in health food stores. My safeway also sells GF cereals, crackers, pancake mix and pasta.

Here's a page with alternative recipes for all those things you mentioned. There are GF cookbooks too.
http://www.celiac.com/categories.php?catid=3400

Here's a page with a long list of GF candy.

It's a little bit inconvenient, but I found narcolepsy and dementia to be much more troublesome.

As to why gluten causes so many problems: because antibodies are not completely specific. They are supposed to bind to the gluten but they bind to other proteins that have amino acid sequences that are similar to gluten. In addition, there are numerous different antibodies that can be produced in response to gluten, so different tissues may be targeted.