It is that simple - we live and then we die...but it's that space in between the two that matters. I've walked away from so many things that are complicated, I actually find that I don't have the emotional capacity to deal with them. I walked away from the last 9-5 that almost drove me to my grave. The money was good, but it wasn't good enough to die for and I couldn't really care what people think. In fact, these days I'm quite happy to think people think lowly of me, it means that I don't have to have any façade and I can just be the authentic me and that way, I won't disappoint myself (not them) by not living up to some ridiculous expectation.
At the end of the day, we're all just people and nobody is better than another by any means really.