I do get somewhat tired of seeing that phrase on Hive, when people try to justify their anti-social behavior.
The down vote is a tool, and I feel it is mis-used far to often, yet when down voters are confronted the response is 'code is law'. I am not anti down vote, they are needed, but they are still misused. And people do not want the algorithm fixed, because they would lose some power.
The definitely does appear to be the case.