Point trajectories make multi-view forecasts more coherent
Esta página aún no tiene una traducción verificada. Se muestra la versión en inglés. Leer la versión en inglés
En palabras sencillas
A video model can make a believable future scene but still show conflicting motion from different cameras. The paper introduces Video-Point Model, which predicts future camera views and the paths of selected scene points together. Following the same points gives the model a shared picture of how the scene moves.
On the reported simulated and real scene tests, the version trained to predict point paths did better on video and 3D motion measures than its video-only version. The finding applies to those tested splits, not every scene.
La investigación
When several cameras record a scene, a future video can look plausible in each view yet show incompatible motion across them. Can a model learn to predict one coherent future event by tracking how the same physical points move through the scene?
Video-Point Model forecasts future camera views and persistent 3D point paths together, using point-motion supervision to anchor visual predictions.
Video-Point Model (VPM) was evaluated on VP-WorldBench, a suite that checks future-video fidelity and whether motion agrees across camera views. In the main test, methods received no future-frame camera parameters and had to infer camera motion from history. Video scores were averaged across output views.
On every reported simulated and real robot and general-scene split, VPM with point-trajectory supervision outperformed its video-only version on all reported video and geometric metrics, with the same training budget.
The joint improvement supports the idea that learning point motion helps produce more faithful multi-view forecasts.
The results cover the reported simulated and real scene splits and robot tasks. Point supervision covers annotated robots and objects rather than the full scene; inference also requires known camera parameters, and processing cost grows with the number of views.
Análisis
Detalles
- Autores del paper original
- Byungwoo Jeon, Sunghwan Hong, Dohyeon Kim, Jonghoon Lee, Jungwoo Park
- Enviado a arXiv
- Licencia
- CC BY 4.0