Do you also have the feeling like you have to urinate very very often, and when you go nothing much comes out? But soon you feel like you need to go again?

Hmm - it's been over 2 weeks since you first posted here - so I think yes it's time for a dr appt.
I think if it was going to turn around for you with home/self care it would have gone away by now.

It really shouldn't be an embarrassing or painful appt.
Most likely just a urine sample needed, and they test it.
Of course if the test comes out negative for infection, then more investigation is needed.