Malicious downvoting has already been defined here as any downvote apart from [1] plagiarism [2] what Community-consensus has agreed is bad behavior.
To move forward we should come up with how the community would reach consensus on what is bad behavior. That can be really hard/impossible to do, so I'm thinking it would be best if we treat each case individually, maybe everytime someone has a reason to downvote another they bring it up to seek consensus and once consensus is reached the downvote can be made with the agreed weight.