I am wondering if it would be better for you to care about how many extra followers you get or your average views. It may also depend on the individual account. For me, I am investing into Steem so but growth potential is screwed when it comes to the amount of SP I can grow per week. For people who have whales constantly upvote them, they could not be growing at all, just have a dedicated whale upvoting them. Yeah, their SP is growing, but is there reach to help others growing at all? I don't think you should base growth off of SP. It should be based off of people reached through your content.
Just my thoughts.
You are definitely right that the most important thing is to have a positive impact on other people. I do find that it goes up and down though, some posts reach a ton of people and others don't and it seems random. In my experience most of the popular users do build up a good stash of SP... maybe that is a coincidence.