Chiosa
Robotics

One robot model learns to predict scenes and control varied robots

2 min
Published on Chiosa Updated
#robot-learning#real-world-deployment#canonical-coordinates#shared-motion-representations

In plain words

Robots can have different joints and record their movements in different ways. This paper introduces GreenWAM, a model that puts those movements into shared body-part coordinates. It learns to predict video, robot motion, or both, and can be adapted to control a robot.

In one RoboTwin comparison, a controller adapted from GreenWAM succeeded more often on average than one adapted from its parent model. That result covers the tested tasks and training recipe; it does not show the same advantage for every robot.

The research

Can a model learn how different robots move well enough to predict what they will see and also produce their actions? Robot datasets use different joint layouts and conventions, so the same action dimension can mean different movements.

The paper tests whether a shared representation can connect prediction and control across those bodies.

GreenWAM shares robot motion coordinates and trains one model to predict video and actions, then adapts it into a controller for different robots.

In RoboTwin, does training a model across robot bodies help when it is later adapted to control a robot? GreenWAM addresses this with shared body-part coordinates and a model trained to predict video, motion, or both, then adapted for action production.

The authors adapted GreenWAM and its parent into single-tower controllers using the same RoboTwin data and 3,000-step recipe. Across 50 tasks, GreenWAM averaged 5.4 percentage points higher success across clean and randomized scenes.

Splitting off a copied action expert added 3.3 points over the GreenWAM single-tower controller, supporting both choices within this tested setup.

These comparisons cover one RoboTwin adaptation recipe, task set, and evaluation budget. They do not establish that the same gains hold for other robots or settings.

Analysis

Deep

Original paperGreenWAM: A World Foundation Model for RoboticsOpen original
Submitted to arXiv
License
CC BY 4.0
Related paperRoboticsReflex helps a humanoid catch thrown boxes with moving-camera vision#robot-learning#real-world-deployment#shared-motion-representationsRelated paperRoboticsHuman motion helps humanoid robots follow instructions#robot-learning#real-world-deployment#shared-motion-representationsRelated paperRoboticsA driving planner organizes candidate paths by motion#robot-learning#real-world-deployment#shared-motion-representationsRelated paperRoboticsA mapless depth-image query speeds reactive drone collision checks#robot-learning#real-world-deployment#gpu-streaming
overviewHugging Face paper page
Next paperReflex helps a humanoid catch thrown boxes with moving-camera vision