DualQuant quantizes language models without calibration data

Low-bit inference is useful only when both model weights and activations can be compressed, but activation outliers make that difficult.

It remains unclear whether a method can choose effective activation rescaling from the model itself, avoiding representative calibration data while preserving quality across modern low-bit formats.

DualQuant derives activation rescaling from weights and jointly optimizes format-aware row and column scales, avoiding calibration data for low-bit inference.

The researchers optimized row and column scales directly against weight-reconstruction error, with projections onto the values allowed by each 4-bit format.

They compared the resulting joint weight-and-activation representations with SmoothQuant, which uses activation samples for rescaling, and with calibration-free baselines across Qwen and Llama models.

The evidence supports the central answer: in the reported tests, calibration-free scaling can remain competitive.

For Llama-3.1-8B in NVFP4, WikiText-2 perplexity was 6.83 for DualQuant versus 7.32 for SmoothQuant, while two other calibration-free methods diverged; a fixed-weight control also showed that the column scale itself reduced activation quantization noise.

The evidence supports calibration-free joint quantization in the tested Qwen and Llama models, formats, tasks, and one packed deployment. The convergence guarantee is conditional on non-degenerate quantization and finite stabilization, and the extended SINQ+ comparisons are best-effort baselines.