Interesting post! Difficult questions indeed.
What separates the simulation of intelligence from the 'reality' of what we call our self-awareness? Are we (our conscious selves) not emergent simulations running on accidental bio-computers merely pre-supposing that we are conscious thinking beings; and how do you determine the answer to that question?
If I'm a conscious thinking being, how do I know you are one too? How do I know I am?
Great points indeed. Consciousness or what we perceive to be our own consciousness may actually be an illusion in itself.