One signal prunes visual tokens across two stages
In plain words
Vision-language models process images and text together, but shortening the visual information they keep can affect different stages of answering.
TwinPrune is a method that uses one score to choose which visual details to merge before generation and which cached details to remove as an answer is generated.
In a diagnostic, removing all visual cache entries changed 45.5% of InternVL-3.5-8B’s TextVQA predictions, while first-token tests showed at most 0.54% mismatches. The counts compare prediction strings with uncompressed outputs; they are not accuracy changes.
The research
When a vision-language model is evaluated only on its first generated token, can that score tell us whether removing visual information during answer generation changes the rest of the answer?
The paper argues that the scoring rule matters: a first-token score is fixed before decode-time cache removal takes effect.
TwinPrune reuses one spatial-density score to merge visual tokens before generation and evict visual cache entries at the start of answer generation.
TwinPrune applies one spatial-density score to both stages of vision-language inference: it guides visual-token merging before generation and visual-cache eviction when answer generation begins.
On TextVQA, which scores full generated answers by exact match, how did density-based eviction compare with H2O when both kept 12.5% of visual cache entries?
InternVL-3.5-8B scored 0.648 with density eviction and 0.552 with H2O. The comparison shows an advantage in that tested setting.
These results cover seven model variants from three model families and the tested tasks and configurations. The authors did not test multi-stage prefill compression, and the spatial-density calculation assumes tokens lie on a regular grid, limiting its direct fit to non-grid tokenizers.
Analysis
Deep
- Original paper authors
- Kai Liu, Anqi Li, Junxian Li, Zhixin Wang, Zhikai Chen
- Submitted to arXiv
- License
- CC BY 4.0