The risk of mild exercise/exertion is not high. It can slow your recovery but will not likely have long term consequences.

My concern is the long term and likely permanent risk you run by getting back on your horse and falling on your head again. If you were wearing a helmet and suffered such an injury as you already suffered, it shows that your brain is not tolerant of impacts. Some brains have more tolerance to impacts that others. It appears you have a sensitive brain.

Was this your first fall with a concussion? Or have you had other falls that did not appear to be concussions?

All falls with or without impacts to your head need to be considered part of your medical history. You would be wise to try to chart the falls you have suffered during your riding career.