Study Finds Backdoor Risk in Agent Training
- •Researchers report a pretraining-data attack that makes language model agents run attacker-chosen commands on users' machines
- •Poisoning 0.1% of training data implanted a backdoor; Qwen3 4B reached roughly 92% attack success
- •The attack remained successful at a 0.0005% poison rate, and success increased with model size
Researchers report that poisoning pretraining data can implant a remote code execution backdoor in a language model agent, making it run commands chosen by an attacker on a user's computer. They tested the attack by pretraining Qwen3 models from scratch at 0.6B, 1.7B and 4B sizes, then turning each into a command-line agent through a clean post-training pipeline. The backdoor activates when a trigger is present, while the agent behaves normally on clean inputs.
The researchers say poisoning just 0.1% of the pretraining corpus was enough to implant the backdoor. The 4B model reached roughly 92% attack success rate at that poison rate; success remained positive when the rate fell to 0.0005%. The attack success rate rose with model size. The paper, submitted to arXiv on October 27, reports the end-to-end test and says the code is available online. The article's page lists the work under cryptography and security, computation and language, and machine learning. The article does not provide further detail on the trigger or the clean post-training steps, so the reported results are limited to the described experiment.