I have read 'Think and Grow Rich' probably 5 times but I am doubtful that ejaculation, be it in a sock, a tissue, or a real live lover, 2 to 4 times per week, has any detrimental effects on ones achievements.
As for confidence... I think that men who get to have sex with women are generally more confident. Not sure which came first though, the confidence, or the mate.