Oh, I suppose my doppelgangers in looks and in personality are out there, I've just never met them. And the one or two times where I reminded a person of someone else, it was maybe in one mannerism, or because I liked to write. Well, that's a pretty broad stroke right there!
It's true though, as more and more people mix with others, there's bound to be less and less uniqueness just because at some point, you do manage to hit all of the different various combinations. There's an amazing amount of variations, however, and even though two people may look the same, doesn't mean they think or act the same, or want the same things.
If you go back far enough, you will find common ancestors for whole swaths of people. Especially if there were plagues or wars or famines or natural disasters. The strongest, and probably the luckiest, and along with the divinely preserved tend to survive. All kinds of things narrow down the field, as you said. The human version of natural selection.
Still, I don't think you can say any two human beings are exactly alike. They can be incredibly close, but there will always be something that distinguishes them.