Researchers introduced ActionOPD, a method for training smaller language-model agents to make decisions like larger teacher models. The paper was published on October 14 and submitted to Hugging Face by Huining Yuan on October 15. It was authored by Huining Yuan, Xiangmin Yi, Zelai Xu, Zixu Wang, Xinlei Chen, Xiao-Ping Zhang, Yu Wang and Chao Yu.

ActionOPD uses semantic actions (meaningful tool-use decisions) as the unit of training. At states encountered by the student model, it samples actions from both the student and a black-box teacher, then rewards the student according to how closely the actions agree in meaning. The approach uses policy optimization (adjusting choices to improve a reward) and does not require the teacher’s token probabilities, matching tokenizers, or reproduction of the teacher’s written reasoning.

The researchers trained Qwen3-4B and Qwen3-8B with two teachers on curated tool-use tasks. Across four out-of-distribution benchmarks—tests on tasks outside the training distribution—the models gained an average of 3.3–6.9 percentage points. ActionOPD scaled with student size and teacher capability and did not require teacher and student to share a model family. On τ^2-bench, it achieved performance competitive with offline distillation and token-level on-policy distillation. The team also explored Self-ActionOPD, using the same model as teacher and adding privileged context.

Researchers introduced ActionOPD, a method for training smaller language-model agents to make decisions like larger teacher models. The paper was published on October 14 and submitted to Hugging Face by Huining Yuan on October 15. It was authored by Huining Yuan, Xiangmin Yi, Zelai Xu, Zixu Wang, Xinlei Chen, Xiao-Ping Zhang, Yu Wang and Chao Yu.

ActionOPD uses semantic actions (meaningful tool-use decisions) as the unit of training. At states encountered by the student model, it samples actions from both the student and a black-box teacher, then rewards the student according to how closely the actions agree in meaning. The approach uses policy optimization (adjusting choices to improve a reward) and does not require the teacher’s token probabilities, matching tokenizers, or reproduction of the teacher’s written reasoning.

The researchers trained Qwen3-4B and Qwen3-8B with two teachers on curated tool-use tasks. Across four out-of-distribution benchmarks—tests on tasks outside the training distribution—the models gained an average of 3.3–6.9 percentage points. ActionOPD scaled with student size and teacher capability and did not require teacher and student to share a model family. On τ^2-bench, it achieved performance competitive with offline distillation and token-level on-policy distillation. The team also explored Self-ActionOPD, using the same model as teacher and adding privileged context.