In plain words
Robots can learn from what happens while they do real tasks, but having people watch and correct every mistake becomes harder as the fleet grows. This paper introduces a way for robots to share experience and improve a common task policy.
When a robot fails, the method uses that situation to find relevant examples of successful actions from the fleet. On Phone Packaging, the trained policy succeeded more often than its starting policy in tests on two robots that had not supplied training data. The tests covered this task and platform.
The research
When robots encounter unfamiliar conditions during real tasks, their failures expose gaps in the policy. But as fleets grow, can those robots improve from deployment experience without requiring teleoperators to monitor and correct every run?
FERT uses robot failures to find relevant successful fleet actions, then trains a shared policy on those actions to improve task execution.
After expert-supervised initialization, FERT uses failure contexts to retrieve relevant successful actions from fleet experience and train the shared policy. People still label episode outcomes and reset environments, while robots supply the action examples.
On Phone Packaging, the selected starting policy succeeded in 79.2% of trials, versus 98.3% after FERT training with 96 robots. Researchers calculated both rates from 120 autonomous trials across two robots excluded from training data collection.
The task results cover five tabletop tasks on one robot platform, and the Phone Packaging comparisons apply to the stated training configurations. The theoretical result concerns a single sufficiently small update under specific assumptions; it does not guarantee repeated learning outcomes.
Analysis
Deep
- Original paper authors
- Chao Yu, Shu'ang Yu, Hao Lin, Hongzhi Zang, Yuxiang Yan
- Submitted to arXiv
- License
- CC BY 4.0