Binary corrections match or improve quality at equal storage

In plain words

The research

What the evidence shows

• A correction with a different role

  BiLoCo (Binary Low-Rank Corrections) is a post-training quantization method for language-model weights. Direct NVFP4 quantization can leave a model-quality gap because its coarse values struggle with some weight and activation outliers.

  A conventional higher-precision correction can address some of that error, but its matrix calculations use the same tensor cores as the main FP4 calculation. BiLoCo asks whether a different correction can use other GPU compute.

• Signs shape the correction

  BiLoCo represents each weight matrix as an NVFP4 residual plus a low-rank correction built from binary sign vectors and scalar coefficients. During construction, it greedily fits signed vector pairs to the matrix, refits their coefficients together, then quantizes the remaining residual.

  At inference, the correction uses additions and subtractions on CUDA cores while tensor cores calculate the FP4 residual. A fused kernel combines both contributions.

• More matrix structure at equal storage

  To check whether the compact factors capture useful weight structure, the authors compare corrections at equal vector-storage budgets on 189 projection matrices from three model families. BiLoCo's rank-512 correction captures a median 47.3–52.9% of each matrix's Frobenius norm across the families, compared with 26.3–29.3% for SVDQuant's rank-32 correction. This is a matrix-level diagnostic, not itself a model-quality result.

• The quality check

  The authors then compare model quality at the same correction-storage budget on five models. On WikiText-2 and C4, BiLoCo's perplexity, which measures next-token prediction difficulty, is lower or tied with SVDQuant's across the tested model-and-dataset comparisons; some results are ties.

  The authors average BiLoCo's values over six sign initializations. The result supports a quality benefit in these evaluations, not a claim that every model or dataset improves.

• Kernel cost and its explanation

  The fused kernel is evaluated separately on an NVIDIA B300 GPU. At batch size one, its correction adds 6–11% latency compared with the same-run padded FP4-only kernel across ranks 16–512.

  In an equal-rank control, binary and dense BF16 corrections take the same time, so the authors locate the binary format's advantage in the higher rank it permits at fixed storage. The latency result is limited to the tested GPU and shapes.