if we do not acknowledge and act on this, we will fall into the trap of limiting ourselves out of convenience
I have always tried to retrospect my life as you said and improvised like you but at times, I feel it creates some sort of anxiety, and needs strong determination to move forward. In contrast, I see almost ninety percent of people around me don't do that and they still live a happy life - so I believe it finally depends on our acceptance. Acceptance of how much we can do and be happy with that probably fits for all..
But listen closely to what they say, and watch how they behave, are they happy?