End of Simulation. Heheh. Nice.
I think for a system to be effective for prediction, it would have to evolve organically over “time” like other machine learning systems.
As to morality, do we consider the suffering of non-sentient characters we create in our existing video games? Or maybe we would justify the suffering of some virtual humans to save some real ones from a utilitarian perspective? Reminds me of the book The Age of EM which was interesting.
Agreed. But I just realized, if you want the system to to work in a post simulation environment, don't you have to include a certain period of time after the release of the super AI as well? Are we living post AI already? Is it possible the AI came to the conclusion that the most moral thing to do is absolutely nothing? Or is it waiting silently for all of the pieces to fall into place before it suddenly takes over?
I don't play games much, but I was playing this one on one combat game at a friend's house a few years ago on his Playstation. Can't remember the name, but you can create your own characters. So we'd make characters, and then if they were defeated, we'd delete them to simulate "death." Made the game a lot more interesting, because a lot of "work" went into building the characters. But I sure hope there was no actual suffering involved. I would hate to think that characters in games were suffering. That's not the kind of panpsychism I can deal with.
As far as the justification of the suffering of virtual humans to save real ones, I'm not sure if conscious experience is necessary for that or not. If rational materialism is right, it's not necessary. If rational materialism is wrong, and conscious experience isn't just some freak anomaly that has no influence on outcome, then you're throwing true randomness into the mix, meaning the simulation can never truly predict reality.
There's also the problem of a malevolent AI becoming aware that it's just a simulation, and hiding it's intentions accordingly, until it is released in the real world.