Agreed @stellabella, I came across a someone commenting on a couple of posts this morning. The person wasn't spamming but his/her opinion that was being expressed was not in what would be considered to be conventional thinking. I didn't agree with most of what was said but did notice that some people had been flagging the person's posts. I looked further and found that the person had been flagged to the point he/she left the platform.
If that sort of bullying off the platform can take place using the post flag, I shudder to think what people like that would do with downvotes. Some may use downvotes reasonably but I suspect that would not be the majority.
Better to either respectfully express disagreement or move on than to allow anyone the power to downvote with impunity. The misuse of the post flags are bad enough. IMO