Before you cancel your appointment, I'd suggest confirming for sure one way or the other what the laws are regarding driving privileges in your area. So I think when you have the discussion with your doctor I’d have a detailed discussion about what their responsibility is regarding reporting.
FWIW, I think that your recovery is important enough to keep the appointment at the brain injury rehab clinic. If you know the exact rules, that decision will be easier to make rather then worrying about what
could happen.
I’m in Arizona, and here the doctors are required to report to the transportation department when they feel that the brain injury is severe enough to compromise a patient’s ability to operate a motor vehicle. I did have to take a behind the wheel driving test to have my license restored. It was a great day in my recovery journey!
Best to you in your recovey.