I think time is entirely subjective and a matter of perspective, just like right and wrong. I think a better question to ask is why do you feel that Karma is so slow to act? Usually impatience is based on our own expectations and if you have expectations that you are in some how "owed" something by the universe, then in time you will get the lesson that corresponds with that sense of entitlement and probably come to the same conclusion that many of us have.
Life isn't fair and our judgments are all subjective and far from perfect. We are not all seeing and all knowing and we can't determine what we or anyone else "deserves" in the grand scheme of things. We can feel that someone deserves to be punished and we can feel that we deserve something that we are not currently receiving, but that is a lesson in and of itself.
To find peace, we have to let go of the illusion that doing good things means good things happen to us and that doing bad things means bad things happen to us. Good and bad are entirely subjective and I wrote a post about that a few days ago if you need more explanation on why that is. To sum it up, just because you think something is good or bad, it's only your subjective opinion and based on your own beliefs. No one owes us anything and we owe no one anything. Simply embrace authenticity and have faith that the universe is going to sort itself out. Even if it doesn't, we have no control over that, only ourselves to some extent.