I honestly can't see what the point is in going to see an orthotist. You need to speak to a sports physio and/or a good swim coach for advice.

I can't say what is wrong (if anything) and, therefore, can't tell you whether continuing to swim is a good idea or not. Sorry, but the Internet has its limitations! Isn't there a swim coach based at the pool where you go swimming who you could speak to (for free) and ask about your injury concerns?

Get a good swimming book or look at swim coaching videos on the Internet if you can't afford to see a coach - it might give you some pointers. Maybe join a proper swim club and get advice there. These things would cost less.

Hope that is a help.