I bet you are catching colds through the three year old. If he spends any time in groups of children, he is bringing home all kinds of disease. When my wife did work at a school, she started coming home with colds.

Using anti-bacterial soaps and wipe also causes the immune system to get weak/lazy. This causes more illness, too.

Research shows that only children have lower immune system strength. Children who share bathrooms with siblings develop stronger immune systems and are less likely to transmit colds, etc.

The important issue for you is to keep you hands clean, especially after touching the 3 year old, and limit how often you touch your face.