A learned drafter predicts which masked position comes next
Cette page n’a pas encore de traduction vérifiée. La version anglaise est affichée. Lire la version anglaise
En termes simples
Diffusion language models fill in masked words a few positions at a time. To keep the model's original choices, a slower method fills one position per pass; filling several at once can change what it writes.
This paper introduces SPOT, which guesses the order in which positions will be filled, then asks the target model to check several possible continuations in one pass. On SDAR-8B tests, it used fewer target passes than a calibrated competing method, though verification can limit the speedup.
La recherche
In block-diffusion decoding, a model chooses both what token to write and which masked position to fill. Exact decoding commits one position per pass, while faster multi-position methods can change the model's trajectory. Can a drafter propose several steps while preserving the target's own decoding choices?
SPOT learns which masked position a diffusion language model will fill next, then has the target verify a tree of candidate continuations in one pass.
SPOT predicts the next position from the target's most recent token probabilities, then builds candidate continuations for the target to check together. This lets it draft several possible steps while retaining a verification rule tied to the target's own choices.
In tests covering math and code tasks, the SDAR-8B target used fewer passes with SPOT than with Spiffy's calibrated draft graph, and SPOT had higher throughput. That measure counts generated tokens per generation-loop time relative to draft-free decoding on the same task, model and temperature.
The speed results come from specific models, tasks and decoding settings. Two transfer tests used an approximate prefix cache and therefore do not establish lossless decoding; verification costs can also erase speed gains, and at some settings on the smaller SDAR-1.7B model Spiffy was faster.
Analyse
Détails
- Auteurs de l’article original
- Eliezer Shlomi, Eliron Rahimi, Avi Mendelson, Chaim Baskin
- Soumis à arXiv
- Licence
- CC BY 4.0