A shared robot policy learns from specialist guidance
VLA-MOPD: On-Policy Distillation for Multi-Task VLA Policy Consolidation · arXiv:2611.12286
In plain words
A robot policy has to turn what it sees and a task instruction into actions. This paper introduces a way to combine several task specialists into one policy: the shared policy tries actions, and the relevant specialist gives it guidance at the situations and intermediate action steps it actually encounters.
On four simulated robot-task suites, the method reached a 98.9% average success-once rate after training, slightly above a method that copied only specialists’ final actions. The result is an average across suites from one training run.
The research
Combining task specialists raises a practical question: does their guidance still help when a shared robot policy reaches situations its demonstrations did not cover?
VLA-MOPD changes the comparison from copying only a specialist’s final action to learning from intermediate guidance at the student’s own states, with a small average gain in a low-data test.
VLA-MOPD combines frozen task specialists into one robot policy by matching their guidance at states and action-generation steps the student reaches itself.
Both LIBERO methods began from a student trained on one demonstration per task and learned from its own rollouts. After 500 steps, VLA-MOPD’s average success-once rate across four suites was 98.9%, versus 98.15% for matching only final actions.
The comparison suggests intermediate transition guidance adds a modest benefit under this low-data setup, where fixed demonstrations may miss the student’s visited states.
The LIBERO comparisons use one training seed, and the robot trial covers only three tasks on one Piper robot. Consolidation also depends on having suitable specialist teachers for the tasks being combined.
What the evidence shows
- What the method addresses
VLA Multi-teacher On-Policy Distillation (VLA-MOPD) is a method for combining task or suite specialists into one flow-matching robot policy. In LIBERO, which tests robot manipulation tasks, the researchers measured success-once: the share of episodes where a task was completed at any point.
The central question is whether a shared policy benefits from teacher feedback at the states it reaches, rather than only from fixed demonstrations or a final-action target.
- Intermediate guidance adds
The near-ceiling scores were close, but the comparison points to a small benefit from matching intermediate transitions in this one-demonstration-per-task setting. It does not show improvement on every suite.
- Feedback and training progress
This comparison tests whether teacher guidance offers more useful feedback per update than a reward assigned only after a full rollout. The recorded run’s higher score at the shared checkpoint is consistent with that explanation, though a single run cannot establish a general convergence advantage.
- Retention has a focused test
At checkpoints with similar Spatial success, Object success fell 3.0 points with reference matching and 13.2 without it. This supports reduced forgetting in this Spatial-adaptation test, though the checkpoints did not have identical Spatial performance.
- A real-robot check
Across three tasks on one Piper robot, the student averaged 90.0% success-once (task completed at any point in a trial), below the routed specialists’ 93.3%. This provides a real-robot result for these tasks, not evidence across other tasks or robots.
Deep
- Original paper authors
- Yongxuan Lv, Zhen Fang, Lichen Man, Hao Li, Pu Li
- Submitted to arXiv
- License
- arXiv non-exclusive