Day, before you invest in too much, start with the shoes. Cheap or old shoes are your enemy. If you haven't already, get fitted by a professional in a solid, sturdy, and proper shoe for your feet.
Your feet, legs, knees, back and shoulders will thank you almost immediately.