Downvotes can be make anonymous I think. Using some shared pool, or something. It can be coded on a blockchain level, dont know the details but I think it can be done.
limit the downvotes to only one free downvote per day, instead of the two now, to prevent downvote abuse
Downvotes abuse can be issue, that is why I propose to lower the free downvotes. Nothing guarantied though.