Part 8/12:
Another flawed assumption is that advanced agents will lack significant inductive biases favoring certain goals, especially when the overarching, distal goals (like maximizing DNA or species survival) are often unconscious and implicit. Humans continuously pursue proximal goals (survival, reproduction) that flow from these long-term aims without explicit consciousness.
4. Cost of Experimentation
The idea that disentangling proximal and distal goals is cheap and straightforward ignores the deep complexity involved—especially when attempting to measure or influence an agent’s inner goal structure.