In plain words
A robot may need a wide camera view to plan where to move, then a close view to grasp or align an object. A fixed way of combining camera views can keep favoring one view as the task changes.
FocusVLA predicts what kind of visual information each upcoming action will need and adjusts the mix of wide and close views. In 50 simulated tasks, its reported average success rate was higher than π0.5’s, though the average does not mean it won on every task.
The research
A robot may need a distant view to plan where to move and a close view to grasp or align an object. It remains unclear whether a policy can shift its use of camera views as those needs change, instead of holding onto an early preference.
FocusVLA predicts future action intent and uses it to route global and local camera information for each upcoming robot action.
Across a manipulation task, a robot’s visual needs can shift from locating a target to close-up grasping and alignment. Can a policy adapt which camera information it uses as those needs change, rather than carrying an early view preference into later actions?
FocusVLA predicts future action intent and uses it to route global and local views for each upcoming action step. On 50 simulated tasks, the authors report average success of 76.92%, compared with 66.82% for π0.5 under the same stated data and evaluation protocols. This supports higher average success in that simulation evaluation, not superiority on every task.
The results cover the tested RoboTwin simulation tasks and six real-world tasks with the reported camera setup. The paper identifies unseen camera layouts and robots with different embodiments as future work, so these evaluations do not establish generalization to those settings.
Analysis
Deep
- Submitted to arXiv
- License
- CC BY 4.0