Chiosa
AI Security & Safety

Signed model weights do not make quantized files reproducible

2 min
Published on Chiosa Updated

In plain words

People often download model files that have been compressed into a lower-bit form. A signature can confirm who released the original weights, but rebuilding the downloaded file also depends on the recipe, calibration data, and computer settings.

The authors checked 198 popular repositories and found that none recorded enough inputs for an independent byte-for-byte rebuild. In specific rebuild trials, supplying missing recipe and build details let many files match exactly. Their statistical checks could still miss a targeted change, so exact rebuilding has limits too.

The research

If a model file is signed, can an independent checker tell whether the quantized file people actually run faithfully came from those signed weights? The conversion also depends on a recipe, calibration data, and computer build, so the signature alone may not identify which output to accept.

The authors measure whether disclosed inputs permit exact GGUF rebuilds and propose a signed manifest that commits to the recipe, calibration data, and build environment.

The authors sampled popular GGUF repositories to ask whether signed full-precision weights let an independent checker verify the quantized file people run.

Because quantization also depends on a recipe, calibration input, and build environment, they scored whether one representative file per repository could be rebuilt byte for byte from recorded inputs.

None of 198 files met that check, and the audited records lacked an environment coordinate. In rebuild trials, supplying missing recipe and build details yielded exact matches for many tested files.

This supports recording the transformation inputs, while the targeted-edit results show why statistical checks cannot guarantee tamper detection.

The measurement covers a popularity sample, and the rebuild and detector experiments cover selected models, quantization types, CPU builds, and edits.

Exact re-derivation checks the quantization step when its inputs are committed and honest rebuilds produce a unique output; it does not establish that the supplied calibration data or source weights are benign.

Analysis

Deep

Next paperNOMOS makes written policies safer for tool-using agents