Actually it is the inflation adjustment. Curation only determines where the reward pool goes, which is out of inflation. If you dont vote, the pool is distributed, just not based upon your stake.
The inflation adjustment happens regardless of what you do, or do not do.