I get the same thing, moreso before when I was spam/scam fighting more. Personally when I see a few flag counts on a successful post I don't think much of it.
Maybe there should be a minimum criteria of reputation or SP for Steemit to allow a downvote transaction from an account (all accounts can always vote on Steem, but Steemit could pick and choose). Though I think it should not be based on number of posts and 100 SP sounds a bit high.
Downvote and flag need to be two different things.