A video model carries scenes and actions across turns
Hy-Nomas: Towards Free-form Interactive World Generation · arXiv:2610.36729
In plain words
Hy-Nomas is a video model that responds to changing text instructions and camera controls over several turns. The challenge is to follow each new command while keeping earlier scene changes and actions consistent.
It keeps lasting scene descriptions available but limits each action instruction to the segment and time when it belongs. In human comparisons on multi-turn text prompts, evaluators more often judged Hy-Nomas better at following instructions than adapted LingBot-World v2. This result covers that test, not every kind of interaction.
The research
When a video model receives a new command, can it follow that instruction without losing the scene or undoing what happened before?
The question is difficult because each new video segment builds on earlier generated segments, where small errors can accumulate and scene details must remain available.
Hy-Nomas generates video from changing text and camera commands, carrying scene context forward while limiting each action to its intended time window.
On a 60-scenario test of multi-turn text interaction, trained human annotators judged Hy-Nomas against adapted LingBot-World v2 for following prompts. Hy-Nomas had a reported 75.0% win rate, a pairwise judgment on cases with two to six turns.
When text and camera controls alternated, Hy-Nomas had higher reported win rates for prompt following and physical plausibility, but a lower rate for camera control. The comparison suggests its strengths depend on what people are judging.
These results come from human judgments on interaction tests; Hy-Nomas trailed Genie 3 on the tested spatial-memory task. The tests do not measure conventional video quality or show that predictions enable successful robot actions.
What the evidence shows
- The challenge across turns
Hy-Nomas is an 8-billion-parameter video world model that generates new video segments in response to successive text instructions and camera controls. The question is whether a model can respond to each new command while preserving the scene and consequences of earlier actions.
That matters because each segment becomes context for the next, so errors can accumulate as the interaction continues.
- Separating scene and action
Hy-Nomas uses structured captions to distinguish descriptions of the scene from specific actions, which are marked with their timing. Its attention rules keep scene descriptions available to later segments while restricting an action instruction to its original segment and time window.
The design aims to preserve context while reducing actions that happen out of order or repeat later.
- Evidence across interactions
The authors report that introducing action-aware attention in later training stages substantially reduced action-order errors and repeated actions across segments. The report is qualitative: it gives no controlled numerical estimate, and the cited description does not establish whether other training conditions were matched.
Human comparisons also favored Hy-Nomas on prompt following in multi-turn text and alternating text-and-camera tests, while its camera-control win rate in the latter test was lower.
- Where results are weaker
The results vary by task. In the spatial-memory test, annotators more often judged Genie 3 than Hy-Nomas better at preserving a scene when returning to it.
The tests assess interaction, plausibility, navigation and spatial recall; they do not measure conventional video quality or show that generated outcomes would support successful robot actions.
Deep
- Original paper authors
- Qiaoling Chen, Jiaxiang Cheng, Xueqing Deng, Tiankai Hang, Ji Hou
- Submitted to arXiv
- License
- arXiv non-exclusive