In plain words
Gemma 4 stores a learned vector for each token at every decoder layer. That makes a large table, and ordinary compression gives each token-and-layer vector its own small code.
OPTIC, a compression method, stores one code for each token and uses a different decoder for each layer to rebuild its vectors. On Gemma 4 E4B, it retained more of the model’s combined score across seven tasks than product quantization at about 100x PLE compression. The result is an aggregate, not a claim about every task.
The research
Gemma 4 stores a vector for each token at each decoder layer, and conventional compression assigns a separate code to each of those vectors. Can a compact representation keep the layer-specific information the model uses without paying that per-layer code-storage cost?
OPTIC shares one code per token across layers and fits the decoders to preserve the vectors’ effect at the model’s projection output.
For BF16 E4B, OPTIC retained 97.8% of seven-task aggregate performance at about 100x PLE compression, compared with 66.3% for product quantization, the strongest conventional baseline. The task-specific scorers measured model outputs; retention compares each task score with its matched uncompressed-PLE score, caps gains at 100%, and combines the seven tasks equally.
On official W4A16 E4B, OPTIC retained 96.9%, compared with 82.8% for the strongest conventional baseline. At actual BF16 E4B compression ratios of 594x and 1,002x, it retained 90.5% and 84.1%.
The evaluation covers Gemma 4 E2B and E4B, and the main comparisons use one fitting seed. The code-sharing comparison also changes how the matched storage budget is divided between codes and decoders, so it does not isolate sharing alone.
Analysis
Deep
- Original paper authors
- Jiacheng Liu, Yaxin Luo, Xiaohan Zhao, Peng Sun, Xinyi Shang
- Submitted to arXiv
- License
- CC BY 4.0