Skepticism is good when it comes to this kind of thing. My gut tells me it's a mix of both "ours" and "theirs" I'm just not entirely sure who they are.
I understand what you're talking about. I've heard that native people of South America couldn't even comprehend what the Spanish ships were when they arrived on their shores in the 1400s because they didn't have a frame of reference.