But your actual reputation is positively affected as perceived by others. In Reality which is not bound by the rule that says a single individual with a large stake can affect your reputation more than many others with lower stake. It is a rule within the code for Steemit which sets those exact parameters for that to happen.
If the code weren't there, you would have no "reputation" score at all.