This is true.
A lot of them seem more or less arbitrary as they will often not bother leaving a comment. I can imagine folks frustration.
But even taking the time to leave a thoughtful comment w constructive criticism, it's a toss up whether the user is going to take it into consideration or worse, they take it personally.
When there is at least one comment that explains it can be assumed that it is not a systematic attempt to harm anyone, but when the votes come from accounts that only cast negative votes and do not publish anything, they can no longer be given the "benefit of doubt"