A visual tokenizer keeps discrete features informative

In plain words

The research

What the evidence shows

• The fidelity question

  The paper studies whether discrete visual tokens can retain the information a multimodal model needs. It introduces Factorized Vector Quantization (FVQ), a method that turns visual features into discrete tokens.

  The authors use downstream multimodal understanding as a proxy for semantic fidelity, then ask whether reconstruction error tracks that proxy and whether adding codebooks can keep improving understanding.

• Error and understanding diverge

  To test whether reconstruction error predicts understanding, the authors compare single-codebook vector quantization with representations retaining alternating groups from an eight-group residual quantizer. Reconstruction error is measured on ImageNet validation, while understanding is averaged across eight downstream tests.

  One partial representation has much higher reconstruction error yet better understanding; another has comparable error but also better understanding. The two contrasts show why reconstruction error alone is an unreliable guide to this proxy.

• Why the bit budget stalls

  The authors find that residual and product quantization gain little when their codebooks increase from eight to sixteen. In their account, later residual codebooks receive weaker leftovers, while product quantization divides features into narrower groups, so each additional codebook contributes less semantic information.

  FVQ instead gives parallel codebooks fixed-dimensional views of the full feature. In the shared setting, its average understanding continues to improve and ultimately matches the continuous Qwen3 ViT representation.

• A larger evaluation

  The authors report state-of-the-art performance among discrete-representation models, with a nine-metric average 22.8% higher than UniAR and 59.2% higher than Emu3-Chat. They also report performance competitive with continuous-representation Qwen3-VL, including on fine-grained OCR tasks.

  These results extend the evidence to a larger training configuration and a broader set of tasks than the shared scaling experiment; the reported comparisons do not establish that all baseline training procedures or evaluation conditions were identical.

• The claim’s boundaries

  The evidence supports claims about understanding on the paper’s evaluated image tasks, which the authors use as a proxy for semantic fidelity. It does not establish fidelity for every possible meaning or use of a visual representation.

  Additional manually selected examples illustrate particular outputs but are not statistical evidence, so they cannot show how often the illustrated outcomes occur.