Robot action selection gains depend on candidate design
How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop · arXiv:2610.37196
In plain words
A world action model (WAM) proposes a chunk of robot actions and predicts what the scene will look like afterward. The paper asks whether choosing the best of several proposed chunks really helps, or whether that chunk only looked best because of luck in what the robot did later.
The authors replay each candidate from the same simulated state several times, choosing with some outcomes and checking the choice on a fresh one. In three RoboCasa tasks, the fresh check preserved only a small share of the gain suggested by choosing the best single run.
The research
When a robot model considers several action chunks, does the best observed one represent a reliably better choice, or did it simply get lucky with what happened afterward? The answer matters for measuring how much choosing among candidates can improve a fixed model.
The authors introduce HeadroomBench, a counterfactual protocol and branching data package that measures which robot action candidates remain better across later outcomes.
The protocol restores the same simulated state for each candidate, repeats the later policy continuation, and chooses candidates using outcomes separate from those used to score them. This tests whether the apparent best action remains better beyond the particular run that made it look best.
In three RoboCasa tasks, an oracle using three outcomes per candidate realized only 9% of the single-continuation gain on a fresh outcome. That result suggests much of the apparent gain came from continuation luck, though the estimate is uncertain.
The selector results come from simulated LIBERO-Plus tasks and do not establish performance on physical robots or other settings. The held-out-task gain was positive on 30 of 40 tested tasks, not every task.
What the evidence shows
- What can selection really improve?
The paper introduces HeadroomBench, a counterfactual protocol and branching data package for world action models (WAMs), which generate robot action chunks and predict their visual consequences. It addresses a basic problem with choosing among candidate actions: the candidate with the best observed outcome may have benefited from random later actions.
The protocol asks how much candidate advantage remains when that later randomness is separated from the choice.
- A fresh check changes the result
RoboCasa and LIBERO-Plus are simulated robot-task suites, where success measures whether an episode completes its task. In a diagnostic on three RoboCasa tasks, the best of 16 Cosmos Policy candidates appeared to raise success from 46.1% to 58.3% across 84 decision points.
Yet an oracle that chose using three outcomes per candidate and was scored on a fresh continuation realized only 9% of that apparent gain. Its confidence interval was broad and included zero, so the point estimate is uncertain.
- Some alternatives are more distinct
The researchers then compared candidates sampled directly from a policy with structured alternatives that shifted its action chunk along robot axes. Across four WAMs and the tested RoboCasa and LIBERO-Plus conditions, at most 22% of the apparent gain for native samples survived held-out evaluation, compared with 67–95% for structured perturbations.
These percentages are shares of the single-continuation gain, not task-success increases. In the two tested models that imagined action consequences, those imagined futures also ranked candidate outcomes about as well as real futures.
- A selector turns headroom into gains
To use the measured headroom, the authors built a selector that predicts when the policy action may fail and, only then, ranks alternatives. On reserved episodes of Cosmos Policy’s training tasks in LIBERO-Plus simulation, the selector raised closed-loop success from 76.9% to 81.6%.
When trained without the test tasks, it raised success from 62.6% to 69.8% on those omitted tasks. The average transfer gain was positive, though 10 of the 40 tasks did not show a positive gain.
Deep
- Original paper authors
- Haojia Gao, Yue Cao, Yumin Liu, Haohua Que, Mingkai Liu
- Submitted to arXiv
- License
- arXiv non-exclusive