I think it would depend on who it is. If it's someone you knew fairly well and who knew your situation it might hurt their feelings. If it's just an acquaintance who doesn't know you well at all it might work.
I don't know why but I always feel the need to explain myself. I'm trying really hard to break this habit because a simple "no, thank you" should suffice. I think it's rude of people to keep on and not take that for an answer. Plus, if you wanted them to know your reasons you'd tell them. Some people just feel entitled to know everything.....
I understand what you mean, though.