In my own opinion, downvoting is fine for me as well. I know it can hurt someone's thoughts but this is to make sure for fair claiming rewards. Although others might find it offensive but it's just like in real life. "Not everyone will like you for sure." But forget about that thought. Just telling you that in real life if you have rewards too much but then it's not worth it. You will hear rumours about you. That's why downvote is necessary into our blockchain.
Don't think someone is mad at you. Just think of it that it's the right thing to do for the blockchain and for others.
that's my understanding about it, I'm not sure if I'm correct. lol