RoleQ aligns ternary weight scales with how model units work

Below four bits per weight, a language model can still predict text acceptably while losing factual knowledge and task accuracy.

The paper asks whether converting an existing checkpoint to ternary weights can preserve more of that knowledge without relying on the costly training procedures used by some alternatives.

RoleQ-T converts existing language models to ternary weights, assigning scales by whether each unit reads from or writes to the shared residual stream.

The authors ask whether existing language models can be converted to very low-bit weights without losing as much factual knowledge and task accuracy.

RoleQ-T combines scales assigned by unit function with a ternary conversion procedure and a brief step that adjusts scales while holding weight codes fixed.

On Qwen3-8B-Base, the authors’ evaluation harness found a 47.6 MMLU score after this step, against 25.9–34.8 for three tested post-training ternary methods at their best available stages. MMLU tests academic knowledge questions and reports the share answered correctly. This result supports the recipe on that checkpoint, although the methods’ rates and stages differ.

These results cover weight storage and selected quality tests on the listed checkpoints. The scale-placement advantage was shown in a rotated basis, and the evaluation does not test generation, reasoning, speed or energy.