It was my first time riding outside of an arena and the horse that I normally rode had to be highly encouraged to even walk let alone do anything dangerous. But I went riding for my first time out in the fields on a completely strange horse in the middle of spring and he has a funny gallop that I was not used to. I have not been riding since because I seem to be so accident prone that it is just not worth it.
I understand what you mean about it being difficult to give up. I have refused to give up skiing and dancing even though it would be easy to get another concussion doing either of these sports. Dancing raises my spirits like nothing else. I could be having the worse possible day, and the dancing could make me feel even worse physically it raises my spirits so much that it is worth it.
Good Luck!