You should try not to become a hermit. However, you should also take care not to push too far.

If I were in your position, I probably would not go just for fun. I might go if anything in my future actually turned on going. This is not advice. I am simply saying what I would do given your choice. Only you are in a position to do the cost-benefit analysis for your own case.