I agree, but even the smaller upvotes carry a weight. I mean it's not merely a monetary gain that you gain from upvotes, but it also helps with your reputation. Small streams make big rivers.
There's something one can do about the "dust" though; A delegation to @dustbunny has been a great help during the past years, as it can give a small upvote to those comments below the treshold, so that the "dust" turns into something tangible.
!PIZZA
Gonna research about dustbunny. I only know dustsweeper