A method builds grounded tool-use training interactions
Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents · arXiv:2610.33568
In plain words
Tool-use agents learn from examples of conversations and actions, but many examples may be short or may not show how to recover after an error.
This paper introduces Trajectory2Gym, a method that creates interactions in real task environments and keeps an action as a training target only when the agent could justify it from what it had seen.
The authors trained five sizes of Qwen3 model with this data. Their scores for stateful tool-use tasks and function-calling tasks rose at every tested size compared with each model's starting version, though the evaluation has limits, including a seen-task setting for one Tau1 task group.
The research
A tool-use example can end successfully without teaching an agent how a later decision depends on earlier evidence, or whether an action was justified by what the agent could see. In the authors' common corpus, examples with extended decisions and grounded recovery after errors were uncommon.
Can training data be built to cover those dependencies and keep its targets learnable from the agent's own history?
Trajectory2Gym builds tool-use interactions around underrepresented dependencies, checks targets against visible evidence, and weights each interaction equally in training.
The authors trained each of five Qwen3 scales for one epoch on interactions selected and weighted by Trajectory2Gym, then compared it with its own public initialization.
On Tau1 and Tau2, which test stateful tool use, the equally weighted mean of native task-success averages increased at every scale; BFCL's function-calling Overall score did as well.
These paired comparisons support the conclusion that the complete post-training recipe improved scores on the tested tasks under the reported conditions. Matched 8B controls also point to contributions from grounded targets and structural selection, though their small gains are directional.
The paired results cover the tested Qwen3 models and task populations, with each task run once.
Tau1 Airline uses the same task identities during training-side construction and evaluation, so it does not test task-disjoint generalization there; the single-run Qwen3-8B component controls provide directional evidence for smaller gains.
What the evidence shows
- The data gap
Trajectory2Gym is a method for constructing and selecting training interactions for tool-use agents. It targets two weaknesses in the available common corpus: 40.9% of its 34,772 certified interactions had at most two supervised assistant decisions, and 10.0% had a grounded decision after an authentic error.
These corpus figures motivate greater coverage of multi-step dependencies and recovery; they do not describe all tool-use data.
- Construct realized experience
To add missing experience, the method represents dependencies between decisions and observations as compositional paths. It generates tasks for paths with too little support, then executes them in native environments.
A proposed path counts only when its dependencies actually appear in the resulting interaction. This makes the training set reflect experience the agent encountered, rather than structures merely intended by the generator.
- Check each decision
An interaction's success does not make every action a sound target. Trajectory2Gym checks that supervised decisions fit the required workflow, are locally valid, and can be supported by information visible to the agent at decision time.
A rejected action and its authentic error can remain as context, while the failed action is not imitated and a grounded recovery can become a target.
- Compare trained models
The authors trained five Qwen3 scales for one epoch and compared each with its own public initialization. Tau1 and Tau2 measure native task success on stateful tool-use tasks, averaged equally across domains; their equal-weight mean rose at every scale.
BFCL measures function calling across non-live, live, and multi-turn groups, and its Overall score rose at every scale too.
- Inspect two components
Matched Qwen3-8B controls examined two parts of the method. With interactions and visible histories fixed, decision-grounded targets raised the three-test mean by 2.75 points over loose success-derived targets.
At a matched selection and token budget, structural selection covered 942 more canonical paths and raised that mean by 2.32 points, though BFCL slipped slightly. These single-run controls are directional evidence, not additive estimates of the full method's gain.
Deep
- Original paper authors
- Yihao Hu, Zhihao Wen, Yitong Lin, Siyuan Lu, Pan Wang
- Submitted to arXiv
- License
- arXiv non-exclusive