A shared visual code keeps image detail and semantic information
dNaViT: Discrete Codes are General-Purpose Visual Representations · arXiv:2611.12426
In plain words
A visual tokenizer turns an image into a set of reusable codes. The challenge is that one set must keep both the meaning of an image and details such as texture or small writing.
dNaViT learns codes from a frozen vision system and trains them to rebuild both its visual features and the original pixels. In tests, the codes supported image reconstruction as well as understanding and image generation, though some comparisons used different training or scoring setups.
The research
A visual code that preserves meaning but loses small details may fail at reading text or rebuilding images; one that preserves pixels may not support understanding. dNaViT’s results change the expectation that a single discrete representation must choose between those demands, while its comparisons leave open how much downstream performance comes from the codes themselves.
dNaViT turns images into shared discrete codes and trains those codes to preserve both semantic features and pixel detail for several visual tasks.
dNaViT’s codes are trained to reconstruct both visual features and image pixels, then evaluated on reconstruction, understanding, and generation. The reconstruction scores and downstream comparisons show that the shared representation retains information useful across these tasks.
That evidence supports using discrete visual codes beyond image recovery, while the external model comparisons do not isolate the codes’ effect.
The external TokBench and multimodal comparisons use published results with differing training or evaluation conditions, so they do not isolate dNaViT’s contribution. Small text also remains challenging.
What the evidence shows
- One code, two demands
dNaViT is a native-resolution visual tokenizer, a system that converts images into discrete codes for reconstruction and other visual tasks. Its question is whether one finite vocabulary can retain both semantic information used to understand images and the fine detail needed to reconstruct them.
The design trains the same codes against both kinds of target, so the central claim concerns a shared representation rather than separate codes for each task.
- Reconstruction keeps detail
ImageNet scores show overall pixel recovery, while TokBench tests whether text in images is recognized accurately. dNaViT-G64 scores higher than the published DC-AE f=32 result on TokBench, but differing training and evaluation configurations prevent a controlled comparison; small text remains difficult.
- Understanding uses the codes
The tests of visual understanding ask whether the codes also retain useful meaning. In the reported comparisons, dNaViT leads the evaluated discrete-representation models on STEM and general understanding, and is comparable to Qwen3-VL-8B on MathVista.
For text-heavy visual tasks, it leads the compared models on both CharXiv tests and is comparable on DocVQA and OmniDocBench. These are model-level results, so they support capability in the tested settings without isolating the representation as the cause.
- Generation extends the case
Generation supplies another test of the shared representation. On GenEval2, which checks whether generated images match combinations of elements and relationships in text, dNaViT-8B has the highest overall score among models in the comparison that report it.
Taken with the reconstruction and understanding findings, this supports the paper’s case that discrete codes can serve several visual tasks. Differences in external models’ training and evaluation setups limit what the comparisons establish.
Deep
- Original paper authors
- Xiaotong Li, Zhengrong Yue, Xu Huang, Jialv Zou, Minhao Jing
- Submitted to arXiv
- License
- arXiv non-exclusive