It is irrational to think the weight of a falling acorn can cause damage. If the acorn was of the variety with a sharp point or the stem hit you straight on, there might be an extremely minor skin trauma with some annoyed nerves.

An acorn weight 3 to 6 grams. A human head weight 4500 to 5000 grams. An acorn falling from 30 feet, exerts less than 0.12 to 0.25 G's max on a human head. The acorn may sting but will not inflict any movement force to the brain. You exert far more force on the brain turning your head from side to side or up and down as in yes/no.

As I have said many times, walking down a flight of stairs or even sitting down in a chair can exert 2 to 3 G's or more.

The only way to avoid such everyday risks is to stay in bed. There are obvious risks that can be avoided, contact sports and other more aggressive activities.

You have a choice. Live with anxiety which will surely negatively effect your brain and mental health or accept that life is imperfect and hits happen. Anxiety will change your reaction time and possibly put you at greater risk. Being too reactive is also an increased risk as one tends to jerk more violently during a startle.

Hopefully, you can see the value in resolving your anxiety and irrational thoughts.

I hope you can find help to get better in this regard.