A learned drafter predicts which masked position comes next
Questa pagina non ha ancora una traduzione verificata. È mostrata la versione inglese. Leggi la versione inglese
In parole semplici
Diffusion language models fill in masked words a few positions at a time. To keep the model's original choices, a slower method fills one position per pass; filling several at once can change what it writes.
This paper introduces SPOT, which guesses the order in which positions will be filled, then asks the target model to check several possible continuations in one pass. On SDAR-8B tests, it used fewer target passes than a calibrated competing method, though verification can limit the speedup.
La ricerca
In block-diffusion decoding, a model chooses both what token to write and which masked position to fill. Exact decoding commits one position per pass, while faster multi-position methods can change the model's trajectory. Can a drafter propose several steps while preserving the target's own decoding choices?
SPOT learns which masked position a diffusion language model will fill next, then has the target verify a tree of candidate continuations in one pass.
SPOT predicts the next position from the target's most recent token probabilities, then builds candidate continuations for the target to check together. This lets it draft several possible steps while retaining a verification rule tied to the target's own choices.
In tests covering math and code tasks, the SDAR-8B target used fewer passes with SPOT than with Spiffy's calibrated draft graph, and SPOT had higher throughput. That measure counts generated tokens per generation-loop time relative to draft-free decoding on the same task, model and temperature.
The speed results come from specific models, tasks and decoding settings. Two transfer tests used an approximate prefix cache and therefore do not establish lossless decoding; verification costs can also erase speed gains, and at some settings on the smaller SDAR-1.7B model Spiffy was faster.
Analisi
Dettagli
- Autori del paper originale
- Eliezer Shlomi, Eliron Rahimi, Avi Mendelson, Chaim Baskin
- Inviato ad arXiv
- Licenza
- CC BY 4.0