There are nerves throughout your body, and the upper chest is where many, many are concentrated. They lead every which way, to the arms, the ribs, the shoulder blades and to the breasts. Anywhere you have a nerve it is possible to have neuropathy, a disease of the nerve, or neuritis, an inflammation of the nerve.
I have permanent damage to the brachial cutaneous nerve, which branches off of the brachial plexus, below the collarbone near the armpit. This nerve then splits again. One branch runs under the armpit, to the back of the elbow (I believe it joins up again with the ulnar nerve - not positive). Another branch runs along the side of the rib cage, and between the first and second rib. And the third branching leads directly into the middle of the breast. If these are tender you can trace them with your fingers
Insult or injury to this nerve can cause burning pain at the elbow (like you just hit your funny bone HARD), pain and tenderness at the underarm and down the side of the ribcage, sharp pain between the first and second rib(costochondritis) and painful, sensitive breasts. If I wear a bra, it irritates this nerve, and after a couple of hours, maybe less, it has caused attacks of faux angina, with sharp chest pains, breathlessness, cold sweats, nausea and light-headedness.
My heart has been checked out thoroughly on several occasions over the last 3 years, so I don't worry about that, but these attacks are not pleasant. I've given up bras completely except for "absolutely must" events. If I'm stressed, the nerve gets so flared that just the pressure of my arm at my side can trigger an attack, and I've had some when I've been trying to sleep. It's scary to go from relaxed to fighting to breathe so suddenly.

I also have many clothes I can't stand to wear, my arms are very sensitive as well due to the RSD.
There are lots of other nerves in the breast as it is one of our most sensitive body parts. It does sound similar to RSD but I sincerely hope it isn't, and like you say, there has been no trauma.
I hope you find your answer soon. If it doesn't go away, Lyrica is a med that does work well for these types of pain, and you might ask about a compounded cream or ointment with Lidocaine and/or Ketamine . These actually work much better than oral meds as they are absorbed directly into the skin where the pain is located.
As for the Dr who said "Learn to live with it", he should lose his license to practice medicine or be struck with an equally painful condition! Wish they would bring back the stocks - a sentence of at least 3 days for saying stupid, insensitive things to patients sounds right, and would let us all vent our disgust- I'll supply the rotten tomatoes
beth