I think that the reason why we find more children in poor neighbourhoods than in posh neighbourhood is a mind set thing. I have come to notice that the rich people don't like giving birth to more than 3 kids. Once they have 2 or 3 kids, they go for family planning. That's just how rich and enlightened people's mind seem to work.
Whether currently living in a poor or rich neighbourhood, the most important thing is where did the individual live his childhood and did he fight against so many infectious diseases?
Another thing is that having low testosterone levels doesn't always mean infertility. Therefore, those men in poor neighbourhoods might still be having low testosterone levels because of the perhaps unfavorable environment they lived in when they were young, but it might not be low enough as to disturb their reproductive abilities.
I am truly honored to have you come around. Thanks a lot for the wonderful comment!!