Robot maps can predict how untouched drawers respond to pulls
PhysVLMM: Interaction-Conditioned Physical Belief Maps for Articulated Objects in Robot Scene Maps · arXiv:2610.40539
In plain words
A robot map may show that a door or drawer moved, but not whether a pull will keep it moving or what effort another similar part needs. PhysVLMM adds beliefs about how these parts work to the map.
It keeps uncertainty when one pull cannot reveal everything, and shares pull evidence among similar parts. In tests on real homes, this lowered typical force-prediction error for untouched parts compared with a version that did not share that evidence.
The research
When a robot faces a door or drawer, knowing what it is and whether it has moved does not tell the planner what a pull will do. Can a measured interaction with one part help predict the response of a similar part that has not been touched?
PhysVLMM adds physical beliefs to a robot’s scene map, retaining uncertainty from pulls and sharing measured effort with similar untouched parts.
On real doors and drawers, the researchers compared a map that shares measured effort among look-alike parts with a version that does not. Across 140 untouched parts in 17 homes, sharing lowered typical force-prediction error from 56% to 42%.
Because the comparison added look-alike sharing while using the same inputs, it indicates that evidence from measured parts helped predict the effort needed by untouched parts in these homes.
The force-prediction result covers 140 untouched parts in 17 Hoi! homes. In those homes, averaging all measured parts of the same type performed about as well as sharing evidence within look-alike groups.
What the evidence shows
- From movement to response
PhysVLMM is a method that extends a robot’s Vision-Language-Motion Map with beliefs about how articulated parts respond to actions. The question is whether the map can help a planner decide what a pull will do, including for a door or drawer that has not itself been pulled.
- Keep uncertainty and share evidence
The method keeps a distribution over possible physical parameters when a measured pull cannot distinguish them all. It also groups parts of the same type with nearby, parallel fronts, then shares measured effort evidence within each group. This gives untouched parts information from measured look-alikes.
- Test predictions for untouched parts
On 140 untouched parts in 17 real Hoi! homes, adding look-alike sharing lowered typical force-prediction error from 56% to 42%, compared with a version using a category prior without that sharing. This ablation tests the contribution of sharing measured evidence to predictions for untouched parts.
- A simpler control matches it
A control that averaged all measured parts of the same type in a home performed about as well as look-alike sharing on force prediction and queries. The result supports using evidence from measured parts, while leaving no measured advantage for the more selective grouping in these homes.
- Response results have limits
For 253 held-out pulls across 18 homes, retaining parameter uncertainty lowered final-position CRPS, a score for predicted final-position distributions where lower is closer, from 0.460 with a point estimate to 0.270 of the stroke, but increased whole-trajectory error.
Separately, PhysVLMM’s final-position CRPS was 0.323 cm versus RigPI’s 0.544 cm on 200 shared held-out real pulls, after excluding 42 cases without a RigPI prediction and 21 with training pulls too short for PhysVLMM.
Deep
- Original paper authors
- Dibyendu Ghosh
- Submitted to arXiv
- License
- arXiv non-exclusive