Lactose is the sugar in milk and cream. Something over 40% of adults are lactose intolerant. Cheese is made by allowing specific species of bacteria to "ferment" the lactose in milk. So the big difference between milk and cheese is, milk has lactose while cheese has virtually no lactose.

That said, the reason why people become intolerant of lactose? Lactose is a complex sugar. Lactose is broken into simple sugars in the small intestine by the enzyme lactase. The lactose detection process and the lactase are contained in the villi, the finger-like protrusions which make up the lining of the small intestine. These protrusions are eroded by gluten proteins, and by immune reactions to these proteins. Gluten sensitivity is responsible for a large portion of lactose intolerance, but probably not all of it.

Lactase breaks lactose into glucose and galactose. The intestine can absorb glucose and galactose into the bloodstream, but it can't absorb lactose into the bloodstream. "Yeast" usually means candida albicans, but can mean related yeasts. Intestinal yeasts thrive on sugars like lactose and fructose. When lactose fails to be converted into simple sugars, it stays in the intestine and grows microorganisms, like yeasts.

What to take home? People who are simply lactose intolerant can ingest cheese, but not milk or creme. People who are intolerant of casein can't tolerate any dairy products. Casein intolerance is even more heavily linked to wheat ingestion than lactose intolerance.