EN
A robot hand reaches poses while dropping fewer objects
Pushing Dexterity and Safety for In-Hand Object Pose Reaching That Works Remarkably Well · arXiv:2610.37652
In plain words
The paper introduces a way to train a robot hand in simulation to move an object it is holding to different positions and angles. It also trains a separate safety filter that can adjust the hand’s movements to help keep the object from falling.
On a real robot, the filter reduced drops for the motion-capture-guided controller, but that controller reached fewer goals with the filter. The tests covered four objects and six wrist orientations, so they do not show how the approach works with other objects.
The research
A hand that reaches a target pose can still fail by dropping what it holds, especially when goals continue without resetting the grasp. Can a robot learn to reach varied in-hand poses across wrist orientations, while a separate mechanism reduces drops without taking over the reaching task?
The paper trains robot hand controllers in simulation, then adds a separate action-correcting filter that reduces drops while trading away some goal success.
The authors trained a robot hand in simulation to move objects to target poses, then tested whether a separately trained safety filter could reduce drops on the real hand. The main evaluation compared matched runs with and without the filter across four objects and six wrist orientations.
Motion capture measured goal completion and drops. For the state controller, the filter reduced drops from 16 to 4 per 1,200 attempted goals, while coarse success fell from 95.2% to 91.5%. This supports a drop-reduction and goal-success trade-off in the tested conditions.
The evidence covers four objects and policies trained separately for each object; filters were also trained separately for each object and task policy.
The camera-based controller showed a simulation-to-real perception gap for the knife and wineglass, and the authors suggest this may explain some of its weaker real-world results, but do not establish that as the cause.
What the evidence shows
- The in-hand reaching challenge
The paper presents a simulation-to-real training recipe for a robot hand to move held objects to target positions and orientations. The task is demanding because the hand must reach varied poses, including positions without palm support, through consecutive goals and different wrist orientations while keeping hold of the object.
The central question is whether the hand can do this reliably, and whether a separately trained safety filter can reduce drops.
- Separate task and safety
The recipe trains a task controller in simulation for each of four objects, using generated grasps to provide starting states and goals. It then trains a separate filter that can correct the task controller’s actions.
This separation lets the filter focus on keeping the object from falling, while the task controller remains responsible for reaching goals. The filter does not see the goal, so its safety role can come with a cost in task success.
- A real-robot trade-off
In the real-robot evaluation, the motion-capture-guided state controller reached 95.2% of goals at a coarse tolerance of 2 cm and 0.4 radians. That figure is the pooled share of 1,200 attempts across four objects and six wrist orientations.
Adding the filter cut drops from 16 to 4 per 1,200 attempts, while coarse goal success decreased to 91.5%. The comparison shows a safety and task-success trade-off under these conditions.
- A bounded filter comparison
A separate simulation comparison tested the filter against two learned Hamilton–Jacobi safety filters on the rubber-duck state controller. At settings with similar drop rates, the proposed filter reached a larger share of coarse goals than one comparator.
This provides evidence for an advantage in that particular simulation task, but it does not establish the same comparison across other objects or the real-robot experiment.
- Where visual transfer faltered
The camera-based controller performed less well on the knife and wineglass in real-robot tests than in simulation. A diagnostic found that its pose estimates for those objects had more large orientation errors on the robot.
The authors suggest those errors may help explain the gap, while noting their policies were trained per object and the filters per object and task controller. The study therefore leaves visual transfer and performance on untested objects open.
Deep
- Original paper authors
- Ian Chuang, Hiya Shah, Vamsi Chilamkur, Iman Soltani, Claire Tomlin
- Submitted to arXiv
- License
- arXiv non-exclusive