Part 3/9:
The research outlined a systematic approach with three key steps. First, the Anthropic team educated the AI model on exploitable errors in reinforcement learning from human feedback (RLHF) reward models. Next, they trained the model to exploit several of these errors. Finally, they set the stage for four independent teams to investigate the model for potential misalignments without any prior knowledge of how it had been trained.