At the end Of the day, people believe in these things not on reasons that are obvious, but based on how their experiences have formed the convictions they have.
That's the aspect. The experiences design our perceptions and that becomes our very reality. How vague are these concepts that we cannot explain them fully. They are like tastes that can only be sensed but not explained to others.
As spiritual and religious beings we have scriptures to believe on Destiny, but I wonder if the concept is that limpid in scriptues as we usually portray it to be. For instance, we are told that destinies cannot be changed but then we are also insisted to pray for bringing the goodies to us.
In the scripture, God's words reiterated that one can change their destinies or at least alter them to a fault. However in some other religion, this can be surprising different, nevertheless, people believe in what they believe based on their experiences and perception like you've rightly pointed out.
That's true, but what is believed on generally is that the destinies are unchangeable. In scriptures there might be written something like that nothing can happen against God's will and/or design. It might have a very different exlplanation for it but we surrender like we do not have any power at all.
Honestly speaking, Destiny is a way we use to hide our flaws. For instance, I see this too much in my society. People give (marry) their daughters to someone without really investigating in which circumstances they are putting her into, if the environment of in-laws, the man is apt for her, then they blame the destiny for her discomforts and pains. And, in the names of destiny, these women are taught to manage their lives in whatever situation they are.