A task-building framework checks whether work tasks can be solved
WorkFactory: Turning Professional Intents into Executable Work for Scalable Agent Training · arXiv:2610.37161
In plain words
WorkFactory is a system for making professional work tasks used to train software agents. It starts with the job's intended output, works backward to list the information needed, and checks whether those facts appear in the request or files. A separate solver tries the task; builders fix blocking gaps before a teacher demonstrates it.
In a component study, adding checks and repairs to information planning reduced reported task problems in all eight categories. Models trained on WorkFactory demonstrations also outperformed their matched base models on four text-only professional-work tests.
The research
A training demonstration is useful only if the task gives the solver the premises needed for its work. But if the task supplies the answer or intermediate judgments, it no longer shows the teacher doing that work. How can task builders preserve that boundary while combining information from scattered files?
WorkFactory plans each task's information needs, probes the realized materials, and repairs blocking gaps before a separate teacher solves it.
In a component analysis, information-level planning reduced probe-reported task problems in six of eight categories versus direct artifact planning, but two categories worsened. GDPval-AA tests professional work by having judges compare submissions against the original task requirements; its Elo rating summarizes those pairwise preferences on a calibrated scale.
A student trained on repaired-task demonstrations scored 1,326, versus 1,293 with information planning alone. Probe-reported problems fell in all eight categories after repair.
The component results come from 5,000-task synthesis conditions and the tested student and training setup. Repair-condition defect counts use final probe reports and include tasks later discarded for unresolved blockers.
The broader training comparison covered three model scales in text-only task solving; it did not test learning from visual evidence or reinforcement learning.
What the evidence shows
- Set the task boundary
WorkFactory is a framework for building executable professional tasks for working-agent training. The problem is that a task may contain plausible files yet omit or contradict information needed for its intended work.
Builders must provide the supporting facts while leaving calculations and judgments for the solver, so a teacher's demonstration still shows the work the task is meant to elicit.
- Plan, probe, repair
WorkFactory plans backward from the intended output. Its information support graph records what information the output depends on, where that information should be available, and which results remain for the solver to derive.
After builders materialize the task, a separate probe solver attempts it using only solver-visible materials. A refiner can address blocking support failures, and a fresh probe checks revisions before a teacher receives a retained task.
- Planning helps, with gaps
Researchers compared information-level planning with direct artifact-level planning in two 5,000-task synthesis conditions. A common probe solver assessed the tasks, and researchers counted task-level incidence when a report contained at least one blocker- or correctness-level finding in a category.
Planning lowered incidence in six of eight categories, but leakage or shortcuts and self-inconsistency increased. The graph helps organize support, but planning alone did not prevent every defect after materialization.
- Repair adds evidence
The repair condition added probing and repair after information-level planning. Compared with planning alone, its final probe reports showed lower incidence in all eight defect categories.
Separately, researchers trained the same student model on demonstrations from each condition. On GDPval-AA, judges compare professional-task submissions with the original task requirements; its Elo rating summarizes those pairwise preferences on a calibrated scale.
The rating rose from 1,293 with planning alone to 1,326 with repair. The defect counts include tasks later discarded for unresolved blockers, and the downstream comparison supports the tested process rather than proving each reduction caused the rating gain.
- Training results transfer
For the broader training comparison, researchers trained three WorkPilot models from their corresponding Qwen base models on the same WorkFactory corpus. The WorkPilot models outperformed their matched bases across four professional-work tests.
Together with the component analysis, this supports the authors' interpretation that constructing coherent tasks can improve the utility of teacher demonstrations. The results concern three tested model sizes and text-only task solving.
Deep
- Original paper authors
- Ziyang Zeng, Tingyang Chen, Jia Deng, Qiyao Wang, Shuo Tang
- Submitted to arXiv
- License
- CC BY 4.0