Training video world models for robot learning, not just clearer frames

If a video model is used to teach a robot policy, should it be optimized for accurate frames or for the policy’s ability to learn useful actions?

The answer matters because the model’s job in this setting is to generate imagined experience, and pixel prediction errors do not necessarily reflect which imagined transitions help a policy improve.

DRILL trains a video world model by whether a policy learns from its imagined scenes and then succeeds in the target simulator.

DRILL asks whether a video model used to teach a robot policy should be judged by frame accuracy alone. Pixel errors are counted uniformly, while policy learning depends on visited states and useful actions.

The framework updates a policy from imagined rollouts, then uses that policy’s return in a target simulator to train the world model. In five simulated manipulation tests, DRILL-IMG had higher mean success than WMPO and RLVR-World, although its one-step pixel error was higher than WMPO’s on the same training data. This suggests that useful imagined learning can matter more than pixel accuracy alone.

The reported results come from simulations, and the headline comparisons do not establish that every training condition was identical. DRILL-IMG also uses extra simulator rollouts and computation; the paper does not report tests on physical robots.