It's a hard process which needs some time, but to lose your fear of man is the solution and necessary, anyway I made this experience because I was obliged to do this.
Proverbs 29:25 The fear of man brings a snare,
But whoever trusts in the Lord shall be [a]safe.