World-model rehearsal helps robots retain skills while learning

When a generalist robot learns tasks one at a time, how can it improve the current task without changing behavior it learned earlier, if old trajectories cannot be stored or revisited?

The paper addresses this tension by separating task improvement from the frozen policy and using imagined rehearsal to protect prior behavior.

D2R steers a frozen robot policy and rehearses earlier tasks in a world model, improving new tasks while limiting forgetting without storing full trajectories.

D2R trains a small steering module that changes the noise entering a frozen generative policy. A frozen world model regenerates earlier-task rollouts from initial observations, and a penalty keeps the module close to earlier behavior on those states.

If the model is accurate enough, the new task is learned in imagination; otherwise learning moves to the environment.

In eight-task sequences, D2R raised final average success from 63.7% to 74.8% on CALVIN and from 10.4% to 74.9% on LIBERO. Forgetting, the success lost after a task was learned, was 5.7 and 8.3 points.

Without rehearsal, LIBERO reached 16.9% and lost 78.3 points, supporting the combined steering-and-rehearsal design.

The evidence covers the reported CALVIN and LIBERO sequences and three tabletop tasks on one Franka robot. D2R still stores initial observations to start rehearsals, and learning new tasks in imagination remains dependent on world-model accuracy.