JetSpec++ drafts ahead while the target checks current tokens

Speculative decoding can speed text generation, but a learned draft that uses the target model’s internal states may need to wait until the target checks the current block. Can a system prepare the next block during that wait and still produce proposals the target accepts often enough to help?

JetSpec++ predicts the target’s unfinished verification outcome and refines the next draft in parallel, overlapping drafting with the current check.

JetSpec++ predicts the target states and likely outcomes that are still missing during verification, then prepares possible next blocks while the current block is checked. Its draft model also revises proposals in parallel, using context refresh and anchor replacement between passes.

Across seven tests, JetSpec++ averaged 8.94 committed tokens per verification round versus DSpark’s 6.89 for Qwen3-8B at temperature zero and budget 16.

Since more committed tokens per round can reduce generation work, the result supports the method’s ability to overlap drafting while retaining useful proposals in these tests.

The evaluations cover two target models, seven named tests, and the reported decoding and serving setups. The serving results use single-request runs on GB200 GPUs, and the authors say systems optimization could further reduce drafting overhead; they do not quantify that possible gain.