You are right though, and it does fly both ways, I just feel there should be a medium for people to show that they disapprove of something, as such a dislike button. It would improve metrics, transparency, and overall consensus.
Its a good point about metrics.... the rest of it, the way to indicate that you don't like something without effecting rewards is to post a comment saying "this is fucking stupid"