DuoRate lets MoE models store and read weights at different rates

When a mixture-of-experts model is compressed, how much it stores may not tell us how much data each generated token must read. The router selects only some experts at each batch-one decode step, and those experts can have different precisions.

The question is whether one stored model can account for both costs without keeping a separate artifact for each workload.

DuoRate stores one prefix-readable expert-weight artifact, then uses each workload’s prefill to choose per-expert read depths under a decode-traffic budget.

On Qwen3-30B-A3B-Base, the authors compared DuoRate with other formats using one evaluation harness. At a 2.9 bits-per-weight read budget, its mean accuracy across MMLU-Pro, MMLU, and GSM8K was 76.16%, versus 76.82% for HIF4 at 4.5 bits per weight. On GSM8K’s exact-answer math scoring, DuoRate matched HIF4 at 91.00%.

The comparison shows that, on this model and these tasks, DuoRate’s workload-specific reads can approach a four-bit format’s accuracy with a lower batch-one read rate.

Same-format WikiText-2 controls also found an allocation benefit, while an alternate-objective control exposed a trade-off with stored rate.

The evidence comes from one model and a limited set of evaluation tasks. DuoRate’s serving-efficiency figures are calculated from the stored layout, not measured with an end-to-end prefix-read kernel.