It's still draining rewards out of the pool and therefore hurting the ability to reward real contributions (unless N=0 of course).
I do agree there is no reason to waste downvote mana and take things too far but even a lot of small payouts can add up to a lot. If they aren't adding value then the small amount isn't really a reason to give a pass.
What I thought originally was that specific N=0 case, then thought its better to keep the amount configurable.
Agreed that a small payouts add up and may create a giant snowball at the end.