WorldProbe tests whether video worlds stay coherent as you explore
In plain words
Video world models make scenes that change as a virtual camera moves. A scene may look fine in each image but still change when the camera returns to a place it has already seen.
WorldProbe is a test that lets a software agent steer the camera while models generate video. In tests of ten models, newer ones generally kept local image and content quality usable longer, but ratings of memory and change across longer explorations remained weak. The results cover selected tasks and rollouts up to three minutes.
The research
A video world model can produce plausible images while failing to keep the place they depict consistent. The question is whether its scene still makes sense as a camera explores for longer, moves to new views, and returns to earlier ones. Short clips or fixed camera routes may not expose those contradictions.
WorldProbe pairs adaptive camera exploration with local and long-range scoring to test whether video world models preserve a coherent world over time.
It remains unclear whether a video world model keeps a coherent place as exploration continues, including when the camera moves away and returns. WorldProbe tests this by letting an agent adapt camera actions during generation, then scoring both local frames and consistency across longer spans.
Across ten models, local visual and content quality generally declined, though newer models tended to sustain it longer. Meanwhile, most long-range memory and evolution ratings were around or below the midpoint of a 0–10 scale.
This contrast supports the authors' conclusion that usable frames alone do not show a persistent world.
The conclusion is limited to 164 selected cases, 10 tested models, and task-specific rollouts of up to 180 seconds. The separate longer DreamX example does not extend the cross-model evaluation to that duration.
What the evidence shows
- The persistent-world question
Video world models generate scenes in response to camera or action inputs. The paper asks whether these models maintain a persistent, evolving world as exploration continues, since short interactions and camera routes fixed before generation may miss contradictions that emerge over time or after a viewpoint changes.
WorldProbe is a diagnostic benchmark designed to test that question through extended, adaptive camera exploration.
- Exploration adapts as scenes unfold
WorldProbe contains 164 cases across four kinds of spatial exploration: approaching a partly visible target, following an open path, inspecting a scene from different views, and moving on to new regions. For each case, a goal-conditioned agent chooses camera actions as the model generates new video.
The camera can respond to the scene as it unfolds, exposing behavior that a route chosen in advance might not reveal.
- Local quality has limits
The evaluation separates local appearance and content from consistency over longer spans. Image-quality models score visual appearance, while a vision-language model rates frame content and also compares revisited views and changes across a rollout.
In the ten tested models, local quality generally declined over time, though newer models tended to sustain usable quality longer. The trend describes different models and does not isolate release date as its cause.
- Revisits reveal broader failures
Long-range ratings remained weak across the evaluated models, with most overall memory and evolution scores around or below the midpoint of the 0–10 scale. The ratings reflect scene, layout, entity, and texture consistency across revisits, and whether entities and scenes evolve plausibly in relation to actions.
The authors conclude that locally usable video does not yet mean a model maintains a persistent world during long interaction.
- Evidence has a defined range
The reported cross-model evidence comes from 164 selected cases and ten publicly available models, with task-specific rollout durations reaching at most 180 seconds. The paper also presents one longer DreamX example, but it is not a cross-model evaluation at that duration.
The results therefore support a conclusion about the tested settings, not every scene, model, or longer exploration.
Deep
- Original paper authors
- Minqi Wang, Jie Peng, Hejun Dong, Ruchang Yao, Puyi Wang
- Submitted to arXiv
- License
- arXiv non-exclusive