I do have dairy, Dejibo, just not high-fat dairy. I too need milk. And I am not off grains--I do not have celiac.
Your doctor is absolutely right that it is hard to get a Porph dx when you are not in a flare.
The only thing that is consistent (with or without flare) is the blood enzyme test. For years it was practically impossible to get that test (although I had it during the last years when Dr. Ellefson was running the Mayo Porphyrin labs, so I had the initial benefit of it). Now it seems once again to be a possibility--since Mayo gave it to me without me asking about it, apparently just to cover the waterfront of tests. And it was positive, although not high positive...I was not in a flare.
The only reason I have brought this up with you, as you know, is that you seem to fit a porph profile. I have talked to hundreds of porph patients on line over the last sixteen years, and you fit the profile to me...but that does not mean you have it. I can't dx it. I just suggest that testing is needed. It made a difference to me when it was suggested to me on the Harvard Neurology MS forum. I found out how to avoid triggers and that helped me a lot. Now that I am really old, I guess the MS is asserting itself more than it was for many years, as well as other things like Porph (in flares) and Polycythemia Vera.
I think I've spoken enough about the possibility of Porphyria in you so I won't continue suggesting the tests. But I might suggest it to someone else if I see another person who could have Porph. You are suffering grievously, in my view, and need help. I wish I could help. I can no longer help myself very much as i was able to do years ago.