The thing is, which I know you're aware of, that downvotes are almost never used.
People like you who only post when you have something important to say, no one's gonna care that you selfvote. Others may force themselves to post daily mainly for the self-vote, so the intentions change around a bit where they may not care if content is good, they may not care if anyone's consuming the post, they may not care if anyone else is getting the other 9 votes if 9 other friends of theirs get them instead who are doing the same every day and trading votes with each other so it's basically 10x self-votes per day.
My point is intentions matter more than if you do or don't self-vote or then also vote-trade on top. Then if you try to break that off you're the evil person for getting in the way of a perfect farm and you'll likely get retaliated against, etc.