It sounds like you broke all the rules.
Rule #1-When on SSI/SSDI-don't work(not even part-time)
#2-continue to see your doctor
#3-continue to take your medications

By working while on SSI/SSDI sends out all kinds of red flags to SSA. Plus not seeing your doctor or taking your meds sorta says that your condition has improved. Being awarded SSI/SSDI doesn't mean you have a free pass for benefits. You still have to prove that you are disabled and unable to work.Depending on your age-everyone gets reviewed. Even working part time looks bad.