Robot policies can learn from the future without generating images (F1, F3)
Vision-Language-Action or World-Action? Just Action Models · arXiv:2610.37219
In plain words
Robot policies learn to move from what their cameras see. Some training methods also try to generate what the camera will see next, but it is unclear whether that image-making step is needed to learn from the future.
JAM trains a robot policy to predict actions and features of recorded future observations. It removes the future-prediction equipment before deployment. On one simulated test of perturbed tasks, JAM succeeded more often than a matched policy trained to generate future images.
The research
When robot policies learn from recorded experience, must they generate images of what comes next, or can future observations help through another training signal? Existing approaches bundle this choice with other differences, making their success rates hard to interpret.
JAM trains robot policies to predict features of recorded futures alongside actions, replacing future-image generation while removing that prediction branch for deployment.
To learn from what a robot will encounter next, must training generate future images? Just Action Model (JAM) addresses this by teaching a robot policy to predict features of recorded future observations alongside its actions, without sending those futures into the action path.
On a fixed set of simulated, perturbed robot tasks, JAM succeeded on 79.95% of tasks, compared with 65.27% for a matched policy trained to generate future images.
The shared architecture and training conditions make this evidence that future-image generation was unnecessary for the higher success rate in this comparison.
These results come from simulated robot tasks and specific evaluation protocols. They do not establish how JAM performs on physical robots.
What the evidence shows
- Separate the design choices
Robot policies can learn from actions alone or also learn to generate future camera images. Because these approaches may differ in their pretrained backbone, future-observation objective, and modules used at deployment, a success-rate gap between them does not by itself show which choice matters.
The paper asks whether policies can learn from recorded futures without a visual-generation objective.
- Train on recorded futures
Just Action Model (JAM) is a robot-control training recipe. It pairs a visual backbone with an action expert and trains them with action supervision plus a prediction task: current-observation features predict features of recorded future observations.
A frozen future anchor supplies those target features. The anchor and prediction head are discarded after training, leaving the action path for deployment.
- Compare future objectives
The main test compares Ovis-U1 policies on LIBERO-Plus, a simulated collection of perturbed robot-manipulation tasks where success means completing a task. The policies share their architecture, training data, optimizer, budget, and deployment path; JAM replaces future-image generation with future-feature prediction.
Across the fixed 1,506-task subset, with one rollout per task, JAM succeeds on 79.95% versus 65.27% for the image-generating policy.
- Check action-only training
A separate Qwen3.5-2B control compares action-only training with JAM’s action-plus-foresight training. Its micro-averaged success rate across LIBERO-Plus tasks rises from 77.03% to 81.14%, but action-only training does better on robot-state and language perturbations.
The source does not specify whether both conditions started from the same checkpoint, which limits what this control isolates.
- Apply the recipe across families
JAM is also evaluated with generative, self-supervised, and vision-language backbones. On the same LIBERO-Plus subset, the reported task success rates for Ovis-U1, DINOv2-L, and Qwen3.5-2B are 79.95%, 70.39%, and 81.14%.
This demonstrates use across these families, but their different architectures and action interfaces mean the score differences do not isolate pretraining effects.
Deep
- Original paper authors
- Yonghao Yu, Lang Huang, Runyi Li, Toshihiko Yamasaki
- Submitted to arXiv
- License
- arXiv non-exclusive