Parallel draft correction raises decoding speed without sequential repair
En palabras sencillas
Large language models can draft several future tokens in parallel, but those guesses do not know which earlier guesses will be kept. The paper introduces DCOCO, a lightweight corrector for these parallel drafts.
DCOCO keeps several candidate tokens at each position, scores how neighboring candidates fit together in one batch, and picks a left-to-right prefix. In tests on four target models that check draft tokens and ten suites, it produced the highest average speed under the reported settings, including greedy and stochastic decoding. This measures serving speed, not answer quality.
La investigación
Parallel drafting is attractive because it avoids step-by-step token generation, but lost dependencies can reduce how many proposed tokens the verifier accepts.
It remains unclear whether contextual correction can recover that acceptance without making every drafting cycle pay the latency of a sequential correction module.
DCOCO scores neighboring candidate tokens in parallel, then selects a prefix for the target model, or verifier, to check: the accepted prefix is the longest consecutive run before the first rejection. This improves accepted length and end-to-end throughput without sequential correction.
Using one DFlash pass, DCOCO retains 16 candidates per drafted slot, scores adjacent transitions in parallel, and selects them left to right.
The evaluation compared it with vanilla decoding, DFlash, DSpark, and Domino on four verifier backbones and ten suites; throughput means generated output tokens divided by request latency, including prefill.
For Qwen3-8B without thinking at one stream, DCOCO averaged 959.7 output tokens per second under greedy decoding, versus 839.3 for Domino and 265.6 for vanilla decoding.
It also accepted longer prefixes while correction cycles were shorter than the strongest correction baseline, linking the speed result to both proposal quality and overhead.
The paper trained each method once, and its comparisons use native rather than strictly matched drafted-token configurations.
The results therefore establish the reported advantage only under the tested hardware, fixed suites, decoding settings, and model conditions; they do not measure answer quality.
Análisis
Detalles
- Licencia
- CC BY 4.0