In plain words
Human demonstrations show how fingers move, but matching motion alone can miss the touch that stops an object from slipping.
Contact-Anchored Retargeting for Dexterous Manipulation (CARD) turns those demonstrations into robot motions while tracking where fingers press, in what direction and with what force.
On video examples, CARD's robot hand matched the demonstrated touch points much more often than three motion-matching methods. That checks whether the hand reaches the right contacts, not whether every task succeeds; on real hardware, CARD also uses fingertip force feedback while replaying.
The research
Can a robot reproduce where and how a human hand presses when its fingers differ from the person's and a recording does not provide an object model? Matching hand motion alone can still miss contact, leaving a replay that hovers over, slips on or crushes the object.
Contact-Anchored Retargeting for Dexterous Manipulation (CARD) turns hand demonstrations into robot motions by preserving finger contact locations, push directions and force targets.
In 20 phone-video clips retargeted to the same SHARPA Wave hand, CARD's contact rate was 61.4%, versus 7.6–9.7% for three motion-matching methods.
Contact rate is the share of individual target-in-frame entries where the target lay on or within 2 mm of its assigned robot link and the pad's nearest point touched on the palm side.
All methods were scored against the same stored targets, so this comparison shows CARD more often reproduced the demonstrated contact points in this video test.
The video result measures pose-to-target contact using contacts inferred from video, not task completion. Retargeting quality remains bounded by the demonstration, and open-loop replay cannot recover from a large object displacement.
What the evidence shows
- The missing part is contact
Contact-Anchored Retargeting for Dexterous Manipulation (CARD) is a method for turning human hand demonstrations into robot motions while preserving intended contacts and forces.
It addresses two related gaps: collection interfaces can constrain how people move, and matching only finger positions can miss contact dynamics that determine whether fingers slip, hover or crush an object.
- One interface for touch
CARD converts each demonstration into a shared 3D hand representation with contact targets attached to hand regions. A target records where the hand touches, the push direction and the force.
For each frame, the method searches for a robot-hand pose that reaches those targets and aligns the pad with the push, without needing an object model or simulator. Video forces are synthesized from predicted contacts rather than measured directly.
- Pose scores expose the gap
To compare retargeting, the paper mapped each method to the same 22-joint SHARPA Wave and graded it against identical stored contact targets. Contact rate is the share of target-in-frame entries where the target lands on or within 2 mm of its assigned link and the nearest pad sample is on the palm side.
On 20 phone-video clips, CARD reached 61.4%, while the three kinematic methods scored 7.6–9.7%, showing a large pose-level contact gap.
- Force feedback changes replay
A robot pose that matches a target does not guarantee that the physical fingers apply the intended force. CARD's tactile tracker compares fingertip force with the target, adjusts finger curl and applies torque during replay.
On the SHARPA Wave, the same trajectories were replayed with and without this tracker across three tasks, 10 trials each: success counts changed from 1 to 8 for hammer lifts, 3 to 10 for cup lifts, and 0 to 7 for mustard dispensing.
- The representation can transfer
The paper maps CARD to the SHARPA Wave, Wuji Hand 2, Unitree Dex3-1 and a parallel-jaw gripper using the same contact anchors, cost groups and weights while changing the semantic map for each robot. It shows hardware rollouts for the three dexterous hands and retargeted demonstrations for the gripper, but reports no numerical performance aggregate across embodiments. This supports mapping the representation across robot forms, not equal performance on each.
Deep
- Original paper authors
- Cunxi Dai, Botao He, Francois R Hogan, Wei Pan, Rao Fu
- Submitted to arXiv
- License
- arXiv non-exclusive