MACaQ keeps four-bit quantization within each layer
MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations · arXiv:2611.10649
In plain words
Neural networks can use less memory and computation when their numbers are stored with fewer bits, but rounding those numbers can hurt their output. MACaQ is a method for turning both weights and activations into four-bit numbers, one layer at a time.
It reshapes each layer’s inputs and adjusts the weights, then improves the rounded values using information about the layer. On one Llama-3-8B text test, MACaQ had a lower perplexity score, meaning better text prediction, and took less quantization time than MR-GPTQ under the reported settings.
The research
Four-bit quantization can reduce the precision of a neural network, while transformations that improve quantization can tie neighboring layers together. MACaQ offers a way to keep the work local to each layer.
Its reported Llama-3-8B comparison suggests that this design can preserve text-prediction quality while reducing quantization time in that setup.
MACaQ quantizes each neural-network layer independently, combining online input transforms with Hessian-guided adjustments to its four-bit weight codes.
For Llama-3-8B, the authors compared MACaQ with MR-GPTQ on WikiText-2 text prediction, measuring perplexity and quantization time. MACaQ scored 6.91 perplexity in 88.1 seconds; MR-GPTQ scored 7.11 in 319.1 seconds. Lower perplexity means better prediction on this test.
The result supports a better quality and time trade-off for MACaQ in this configuration. A cumulative test also found lower perplexity after adding its components to round-to-nearest quantization, from 8.28 to 6.87.
The numerical comparisons cover specific models and settings, and the calibration configurations for MACaQ and MR-GPTQ were not identical.
MACaQ’s layer-local design cannot compensate for quantization errors across layers; its online transforms added about 8% overhead on the smallest tested model, Qwen3-0.6B.
What the evidence shows
- Keep quantization layer by layer
MACaQ is a post-training method for quantizing neural-network weights and activations to four-bit MXFP4. It addresses a practical tension: transformations can make values easier to quantize, but propagating them across neighboring layers creates dependencies. MACaQ instead keeps each transformation and quantization decision within one linear layer.
- A faster result in one test
On the WikiText-2 text-prediction test, MACaQ had a lower perplexity score, indicating better text prediction, and took less time to quantize than MR-GPTQ. This supports a better quality and time trade-off for this Llama-3-8B configuration, though their calibration budgets differed.
- Components build on rounding
In the Llama-3-8B component test, adding the method’s components in sequence improved WikiText-2 perplexity over round-to-nearest quantization. The cumulative comparison supports the combined method in this setup, but does not isolate each component’s effect across all combinations.
- Runtime cost depends on model
For Qwen3-32B at batch size 1, disabling online scaling and rotation slightly lowered the measured decoding rate. This indicates a small cost from these transforms in that comparison; the paper reports a larger cost for its smallest tested model, Qwen3-0.6B.
- Evidence beyond language models
The method also transferred to one image diffusion model. For FLUX. 2 Klein 4B, MACaQ’s quantized images were closer to the original BF16 outputs on all four reported fidelity measures than the official NVFP4 version, across the tested prompts and generation settings. This offers evidence beyond language models while remaining specific to that model and setup.
Deep
- Original paper authors
- Lorenz K. Muller, Philippe Bich, Chiara Boretti, Hyun-Min Chang, Jiawei Zhuang
- Submitted to arXiv
- License
- arXiv non-exclusive