As I understand the Fermi Paradox, part of it is also the question why can't we see any evidence of the aliens, not only why haven't we met them in person. For example, we haven't we heard any alien radio signals. Why haven't we detected any Dyson Spheres/Swarms around any stars?
Getting to other stars will be quite unpopular unless some sort of FTL drive is possible, that is obviously a problem. But it is also solvable by things like generational ships, cryostasis (even if you send just embryos that would be later raised on the planet once the infrastructure is set up by machines).