90/30 Club (ML reading) #28: Sample Size Consistency in LLM Poisoning

​Week 28: Sample Size Consistency in LLM Poisoning

​A small number of samples can poison LLMs of any size

A recent joint study by Anthropic, the UK AI Security Institute, and the Alan Turing Institute identified that as few as 250 samples are enough to backdoor LLMs across different sizes (600M to 13B parameters). The experiments demonstrate that attack success depends on a small, fixed number of poisoned examples rather than a fixed percentage of the training dataset, with this vulnerability persisting regardless of how much the training data scales with model size.

​The study's primary focus on pretraining poisoning explores this constant-number dynamic while also demonstrating that similar vulnerabilities apply during fine-tuning. For organizations across various industries deploying AI solutions, these findings imply the necessity of developing new effective strategies against backdoor threats. Authors note that continued clean training and post-training methods may help mitigate risks, providing actionable pathways for strengthening security across the pipeline.

​Join us at Mox to explore:

- Why the threshold for successful backdoor attacks may be lower than previously estimated.

- Effective strategies to mitigate security risks while pretraining models with datasets from the public web.

​🔎Analyzed Papers

​​Discussion at 20:00, (optional) quiet reading from 19:00.