Diverse robot demonstrations improve imitation learning
DiscoDemo: Discovering Efficient and Diverse Robot Demonstrations for Imitation Learning · arXiv:2610.32093
In plain words
Robot learners need demonstrations, but collecting successful examples can be costly. Repeating one successful way to do a task may also leave the learner with too few examples of what to do in different situations.
DiscoDemo uses reinforcement learning in simulation to generate several successful ways to complete a task, filters out failures and undesirable interactions, then trains a separate image-based robot learner from the examples.
Across four tabletop tasks, the resulting learners averaged higher success than learners trained from a generator without the diversity objective, both in simulation and on a real robot.
The research
Collecting successful robot demonstrations takes time, while a generator that repeats one solution may leave gaps in what a learner sees. Can simulated demonstrations cover several successful ways to solve a task and help a separate robot learner succeed more often?
DiscoDemo generates varied, successful robot demonstrations in simulation, then uses them to train a separate visuomotor learner by imitation.
Researchers trained the same pretrained visuomotor policy on 3,000 successful demonstrations from DiscoDemo or P-RFCL, a generator without the diversity objective. They measured the fraction of evaluation episodes or robot trials that succeeded, averaging results across four tasks.
DiscoDemo’s policies averaged 62% success in simulation versus 40% for P-RFCL, and 78% versus 39% on the robot. The higher averages in both settings support the claim that its generated data improved downstream success in these tests.
The evaluation covers four tabletop tasks with simple scenes and a modest simulation-to-robot gap. The method also depends on accurate simulation, privileged state, reset access and task-specific success and safety criteria, so these results do not establish how it performs in more complex settings.
What the evidence shows
- Why demonstration coverage matters
DiscoDemo is a method that uses reinforcement learning to generate robot demonstrations, then trains a separate image-based learner to imitate them. The question is how to get useful training examples when collecting successful robot demonstrations is costly and repeated rollouts may cover only a narrow set of successful behavior.
That narrow coverage can leave a learner without examples for states it reaches after its own errors.
- How the generator learns
The generator starts with a small set of successful demonstrations, but uses their states as reset points rather than copying their actions. Its reverse curriculum begins near the end of a demonstration and moves the starting point earlier as the generator learns to complete the task.
A diversity objective then encourages different episode skills to produce distinct behaviors, while rewarding diversity only when a trajectory completes the task.
- From rollouts to a learner
The trained generator produces demonstrations in simulation. The method removes failed trajectories and filters successful ones for undesirable physical interactions, then uses the retained observations and actions to train a separate visuomotor learner.
That learner sees images and robot motion readings, not the exact simulator state used by the generator. This separation matters because the evaluation measures the downstream learner, not the demonstration generator as a robot controller.
- What the comparison found
Researchers compared DiscoDemo with P-RFCL, which uses the same generator and curriculum without the diversity objective. Both supplied 3,000 successful demonstrations and trained the same pretrained downstream learner with the same recipe.
Across four tabletop tasks, DiscoDemo’s policies had higher average success in simulation and in zero-shot real-robot trials. The results support the method in those settings, though averages do not show that every task improved equally.
- Coverage and learner success
The researchers also compared variation in trajectories from one fixed starting scene with coverage across the full training dataset. Across seven data-generation methods per task, dataset coverage tracked downstream success at least as closely as fixed-scene diversity on every task.
The authors take this as evidence that broad coverage is a useful indicator of learner performance; the correlations are noisy and do not show that coverage itself causes better performance.
Deep
- Original paper authors
- Minho Park, Kinam Kim, Donghu Kim, Byungkun Lee, Dongyoon Hwang
- Submitted to arXiv
- License
- CC BY 4.0