Part 4/10:
Interestingly, the models' capacity played a role: larger models—up to 13 billion parameters—still required only a comparable small set of poisoned data to be compromised. Despite needing exponentially more data to train, their susceptibility to poisonings remained remarkably consistent and low-threshold.
The Implications for Model Safety and Data Collection
Perhaps most alarming is that this attack efficacy does not diminish as model size increases. Larger models, with trillions of parameters like GPT-4 or similar, might still be vulnerable, provided the attacker can introduce malicious samples into their extensive training datasets.