Rewarding diverse robot successes improved tested shift averages

When task conditions change, a robot may benefit from having more than one successful way to act. But ordinary training rewards mark success without distinguishing among successful solutions. Can a training method encourage broader successful behavior and improve task completion under those shifts?

DRIVE adds a reward for diverse successful robot trajectories, aiming to broaden the task-valid behaviors available during reinforcement fine-tuning.

The authors asked whether explicitly encouraging diversity among successful robot behaviors could improve performance when task conditions change.

DRIVE addresses this by adding a diversity reward only to successful trajectories, compared with standard reinforcement fine-tuning that uses task rewards alone.

In simulation, the authors measured task completion under changed conditions and averaged success across six task families. DRIVE had higher out-of-domain averages for both tested policies than Vanilla RFT. The physical comparison also favored DRIVE across two tasks and three specified shifts.

The results cover specified simulation shifts and a physical test of two tasks under three shifts. The theorem concerns optimal policies under the stated reward-shaping formulation; it does not guarantee that a trained policy will generalize. The method also adds computation and relies on vision-language representations.