Only two agent-training methods beat GRPO on average
In plain words
This paper compares ten ways of training text agents that do tasks, such as finding products or answering questions using retrieved information. It asks whether these methods reliably improve on training from task results alone.
Researchers tested them on four tasks and three model sizes, tracking both final results and progress during training. Only two methods had a higher average final score than the reference method, and one of those did not win in most settings. Strong final results could also follow a sharp drop during training.
The research
When many training methods claim to improve text agents, do their gains hold up under direct comparison? Earlier evaluations often reused baseline scores from different conditions and gave less attention to training stability and compute cost, leaving reported gains difficult to compare.
The authors compare ten self-distillation methods with GRPO across twelve settings, measuring final scores, training progress, and compute cost.
The researchers trained ten methods and GRPO in the same twelve model–task settings, then compared repeated final evaluations. Only PCSD and AgentOPSD exceeded GRPO’s equal-weight mean across the settings.
PCSD’s lead was mixed: it won six settings, lost five, and tied one. Training curves and time comparisons add limits to the result: endpoint leaders were not necessarily strongest throughout training, and extra training time did not consistently bring a higher final mean.
The benchmark covers ten selected methods, four tests, and three Qwen3.5 model sizes. Its results do not establish how the patterns extend to other methods, real-world settings, or deployment with users.
Analysis
Deep
- Original paper authors
- Linhao Zhong, Bozhen Fang, Xuehao Wang, Duochao Shi, Zekai Luo
- Submitted to arXiv
- License
- arXiv non-exclusive