I have always been extremely sensitive to mold, I have asthma. Like you I can detect the odor before anyone else and I start coughing and my throat starts to swell shut.

I would check the basement under the family room to see if water is leaking in and the odor filtering up.

It may be even outside. In our last house, if you opened the windows on the shady side of the house, I detected a smell of mold from the soil outside. My asthma is so bad, I have to keep the windows shut and the air conditioner running as soon as the heat turns off and until I have to turn it back on in the fall, just to keep the air free of pollen and mold from the outside. You may also want to have your air vents cleaned professionally. I have them cleaned every few years. I also have those 3M air filters that are suppose to filter out 95% of pollen, mold and air particulate. With all that, I do not have a problem anymore. I hope you find the cause.

kathie