I agree that flagging / down voting is contentious and is likely to remain so. You can't escape the human psychology of it, though I do think a UI change could go along way (not much support for that at STINC at the moment though).
If there were a better way I would support it am I open to ideas.