In plain words
When a model answers several questions about an image or video, it may need different parts of that input for each question. Pruning speeds the first response by processing only some parts, but discarded parts cannot help with later answers.
Raincheck keeps the unprocessed parts and adds them to the model’s context in chunks while it generates an answer or afterward. In tests on two models and image dialogues, it improved average scores over pruning alone, though not every turn improved.
The research
When a conversation includes several questions about an image or video, the parts relevant to the next question may differ from those selected for the first.
Can a model process fewer tokens for its initial response while keeping the rest available to answer the current question and later follow-ups?
Raincheck keeps pruned multimodal tokens and processes them in chunks during generation, improving tested dialogue scores over hard pruning.
Can a model respond sooner by processing fewer image or video tokens first, yet keep information that later questions may need? Raincheck is a multimodal inference framework that preserves unprocessed tokens and gradually adds them to the model’s context during answer generation or afterward.
In matched image-dialogue tests of two models and four pruning methods, Raincheck recovered an average 77.7% of the aggregate MultiVerse score loss caused by hard pruning. A separate video-dialogue evaluation also found higher overall scores than matched hard pruning. The results support deferred recovery in these tested settings.
These results cover two model architectures and specific image and video dialogue evaluations. Video category scores moved in mixed directions, and system timing was measured with one request at a time; the findings do not establish the same results for other architectures or larger batches.
What the evidence shows
- Keep omitted context
Raincheck is a framework for multimodal model inference: an existing pruning method chooses which image or video tokens to process first, while Raincheck keeps the rest available. The problem is that a selection made for one question may omit information useful for the answer or a later question.
The framework asks whether processing fewer tokens initially can reduce the wait while still recovering that context.
- Selections change by question
To check whether question-specific selections differ, researchers compared MMTok’s choices for the first two questions in 48 MultiVerse image dialogues, retaining 30% of image tokens. On average, 19.34% of tokens selected for the second question were missing from the first selection for Qwen3-VL-8B; for Qwen3.5-9B, the figure was 17.32%. This diagnostic demonstrates selection differences, not that every missing token is necessary.
- Recovery improved image scores
Raincheck processes the selected tokens first, then restores the remaining tokens in source order in chunks during generation and, if needed, afterward. In matched tests on two models and four pruning methods, it recovered an average 77.7% of the aggregate MultiVerse score loss from hard pruning.
A separate ConvBench evaluation also found higher mean scores for all eight model-and-method pairs. Both results concern judge scores in the tested image dialogues.
- Video and timing results
The video evaluation extended the comparison to MT-Video-Bench, where a judge checked answers against dataset labels. Raincheck’s overall score was higher than matched hard pruning for both models, though category-level changes were mixed.
Timing on MultiVerse also showed lower mean first-token latency than full prefill, alongside small increases in average time per output token. These measures come from separate evaluations and settings.
Deep
- Original paper authors
- Wenjie Huang, Jun Zhang, Hongyaoxing Gu, Haoyang Huang, Tianqi Xu
- Submitted to arXiv
- License
- CC BY 4.0