Hi @intellihandling!
You're right, the distance could be another problem. Maybe they came thousands of years ago, but given the distance from their natal planets, a return would take thousands of years more. Maybe that's why we don't see any evidence of them.
Very good theory
Thanks for commenting 