Robot touch training improves control with and without touch
In plain words
Robot hands often have to judge contact from pictures, where it can be hard to see pressure or slipping. This paper introduces Spark-0, a robot-control model trained with touch as well as video.
Touch teaches the model where contact happens, and can also help it choose actions when sensors are available.
On five real-robot tasks with live touch, Spark-0 had a higher average success rate than the strongest comparison system. The result applies to those tested tasks and robot setup.
The research
When a robot manipulates an object, images may not clearly show where its hand is touching or whether the object is slipping. Can touch help the robot learn better visual representations, so the benefit remains when it later acts without tactile input?
Spark-0 uses touch to guide visual learning during training and to condition actions when available, with one model supporting deployment with or without touch.
Can touch teach a robot to understand images better, even when it must later act without tactile input? Spark-0 uses contact maps and geometry features to guide visual learning during training, while a separate pathway can pass live touch to its action generator.
On Ego Humanoid, which tests task completion across four visual and time-horizon settings, the model trained with 1,000 hours of touch averaged 71.72% success without touch at evaluation, versus 52.45% for the vision-only pretraining variant.
The shared RGB data, compute and downstream budget make this evidence for a contribution from tactile pretraining.
The comparisons cover specific robot tasks and training setups. The Ego Humanoid result does not establish why touch helps, and neither evaluation establishes performance on tasks or robots beyond those tested.
Analysis
Deep
- Original paper authors
- Tianxing Chen, Wenwei Lin, Chuqiao Lyu, Peicheng Xiang, Junyuan Tang
- Submitted to arXiv
- License
- arXiv non-exclusive