QUILT speeds long-context prefill by sharing query work

As context windows grow, prefill must process more selected key-value data before the first token appears. It remains unclear whether neighboring queries contain enough shared work for a kernel to reuse it, rather than repeatedly loading, converting and computing the same entries.

QUILT groups neighboring sparse queries and reuses their shared key-value entries, turning overlap into lower kernel and first-token latency.

The authors compared QUILT with OPS-Transformer, an optimized sparse-attention kernel, in the same XYServe serving setup. They tested GLM-5.3 and DeepSeek-3.2 on four long-context workloads under tensor and sequence parallelism, and compared their official task scores on 21 datasets.

QUILT reduced the number of processed key-value entries and lowered both kernel latency and time to the first output token. For GLM-5.3, average kernel latency fell 53.8% under tensor parallelism across the four workloads, while the mean absolute score difference across 21 datasets was 0.42 points. Together, these results support faster tested prefill with little measured score change.

The evidence is bounded to GLM-5.3 and DeepSeek-3.2, LongBench tasks, four workloads for performance and first-token latency, 21 datasets for accuracy, and a 16-Ascend-910C-NPU testbed.

It does not establish the same gains outside those models, tasks, configurations, serving setup and accelerator conditions.