TurnWise gives streaming video decisions credit turn by turn
Turns Matter: Advancing Streaming Video RL via Turn-Level Credit · arXiv:2611.06728
In plain words
TurnWise is a way to train video models that must decide, as a video unfolds, whether to answer now or keep watching. It scores each choice by looking at the interaction with later replies replaced by silence, so a reply is not credited for rewards earned earlier. It also gives each answer-or-wait choice equal weight in training.
In one analyzed training group, 432 of 722 repetitive replies that had positive whole-interaction credit instead received negative credit for that turn. This shows the two ways of assigning credit can disagree in that group.
The research
In a streaming video conversation, an answer can help while a later repeat adds little. When training treats the whole exchange as one success or failure, it can blur which choice deserved credit. TurnWise tests whether assigning credit to each answer-or-wait choice changes that learning.
TurnWise trains streaming video models by assigning reward credit to each answer-or-wait decision and weighting those decisions equally during learning.
TurnWise scores each sampled interaction again with later replies replaced by silence, then compares each turn's remaining reward with aligned trajectories. In one analyzed Qwen2.5-VL-3B cohort, 432 of 722 judge-labeled repetitive replies with positive whole-interaction credit received negative turn-specific credit. This shows a shared signal can conceal differences between decisions.
The main comparisons cover two Qwen backbones on three test suites; the detailed credit analysis uses one Qwen2.5-VL-3B training cohort. Performance beyond those settings was not measured here.
What the evidence shows
- Credit each streaming choice
TurnWise is a reinforcement-learning framework for training streaming video models to answer or wait as observations arrive. It addresses a mismatch: a whole-interaction reward can give a later reply credit for an earlier success, or penalize a useful follow-up after earlier failures. TurnWise instead assigns a learning signal to each decision.
- A reply can lose inherited credit
The comparison shows that global credit can favor a repetitive reply because of rewards earned earlier, while turn-specific credit can assign it a negative learning signal.
The two credit assignments can therefore direct learning differently within the same interaction; this finding applies to the analyzed replies, not all replies.
- The result spans three test goals
Across three tests covering answer quality and timeliness, event-triggered response timing, and correctness from observed video prefixes, TurnWise had the highest aggregate scores among the reported methods for both evaluated Qwen backbones.
The range of test goals supports a broader reading than any single score, while the evidence remains tied to those models and tests.
- Equal weighting changes repetition
In a Qwen2.5-VL-3B component comparison, adding equal turn weighting to turn-specific credit reduced repetition among non-silent replies from 15.53 to 5.98.
This comparison isolates the reported component contrast, but does not establish that the same change occurs with other backbones or training setups.
Deep
- Original paper authors
- Shuai Dong, Kanghui Tian, Linghan Chen, Wenqi Shao, Hongrui Wu
- Submitted to arXiv
- License
- arXiv non-exclusive