Thank you for the clarification. I now see more clearly where AI is a potential impending disaster. As you wrote there is shortfall between expectations and implementation. This may lead to AI being released from the labs in commercial or military products that could fail catastrophically and reveal those very flaws that we missed, at great cost, financially or in human lives.
I agree completely with your concluding remarks, how can we possibly create a complete General AI when we don't even fully understand our own organic intelligence and consciousness?
Yes good point, it can backfire on us. We are trying to create something that can lead to our destruction. Much fiction has used that theme. From Terminator to the Matrix.
The thing is, I think many military heads have also been watching and reading the same fiction and are using their bottomless budgets to fund the development of their dream weapons, without a thought for how it could go so horribly wrong. They seem to ignore the moral or the warning of such fiction.
Commercially also, businesses see the dollars and not the problems.
We seem to become mesmerised with challenge of, we can, to the detriment for any consideration as to why we perhaps should not.
Then there the unforeseen consequences that none of us can anticipate.