"decentralized" means there won't be rules to ensure things are "free" or "uncensored", no less fair.
Downvotes are vital to the algorithm that determines how valuable the community as a whole (including each member with their own stake in the platform), deem a specific post.
If you click the flag button, what's the very first reason it gives for potentially flagging a post? "Disagreement with rewards"
This means anyone, for any reason, can upvote or downvote a post. Flags should not be "completely removed"