EN
A shared spatial map helps robots coordinate across views
World Dreamer v5-lite: A Multi-Robot World Action Model with One Dreamer · arXiv:2610.37518
In plain words
Robots working together may need to use what another robot can see. World Dreamer v5-lite is a system that puts camera information into a shared map of the workspace, then predicts separate actions for each robot while letting their information interact.
In a multi-robot simulation test, it had a higher average task success rate than the strongest evaluated comparison across three tasks. The paper also reports results from household tasks and four physical robot tasks, so the evidence applies to those tested settings.
The research
When several robots work in one space, each may need information visible only to another robot, yet their camera views do not automatically line up and their controls may differ. How can a team share useful visual context while keeping each robot’s actions distinct?
World Dreamer v5-lite gives each robot a shared spatial view of the workspace while predicting separate actions, then tests the approach in simulation and with physical teams.
World Dreamer v5-lite combines calibrated camera views into a shared spatial state, then lets each robot query that context and exchange information with teammates while keeping its own action stream. During training, selected views are hidden but every robot’s action target remains.
On RoboFactory’s three simulation tasks, researchers found a 74.0% unweighted mean task success rate, compared with 50.3% for π0.5. Results on household tasks and physical robot trials also show higher reported mean success than the strongest compared methods in those settings.
These results cover three RoboFactory tasks, four real-robot tasks and 50 of RoboCasa365’s 365 tasks. The comparisons do not establish performance across other tasks or robot teams.
What the evidence shows
- The coordination problem
In a shared workspace, each robot’s cameras capture only part of the scene, and images from different cameras do not automatically correspond to the same places. At the same time, each robot’s actions depend on what its teammates are doing.
The paper asks how a team can use observations across robots while preserving controls for robots that may have different bodies and action spaces.
- Share context, separate actions
World Dreamer v5-lite is a multi-robot manipulation framework that predicts an action chunk for each robot. It uses calibrated camera geometry to place visual features in a shared representation of workspace locations.
Each robot’s query reads that shared state and exchanges information with other queries, while each robot retains its own action stream and proprioception.
- Train with hidden views
During training, the system hides selected camera views from both visual paths but retains the demonstrated action target for every robot. This creates examples where a robot must predict its actions with incomplete visual coverage, while remaining views can provide context.
The design encourages information sharing, but the reported comparisons do not isolate its effect from the rest of the system.
- Compare task success
RoboFactory tests multi-robot teams on three simulation tasks: placing food, aligning a camera and taking a photo. Researchers report task success averaged equally across the tasks.
World Dreamer v5-lite’s mean was 74.0%, versus 50.3% for π0.5, the strongest evaluated baseline; all methods used the same modified camera configuration, though other training conditions are not established as identical.
- Check other settings
The paper also reports household manipulation results and physical robot trials. On RoboCasa365, the model scored 58.08% across 50 tasks, including 30.6% on 16 Composite-Unseen tasks absent from its Human300 training data.
Across four physical robot tasks, its mean success rate was 78.8%, compared with 65.0% for π0.5. These results extend the evidence to those settings, not to every household task or robot team.
Deep
- Original paper authors
- Guoning Wu, Zijian Cai, Yuhang Zhang, Ying Liu, Yongbin Zheng
- Submitted to arXiv
- License
- arXiv non-exclusive