Chiosa
Machine Learning

A learned drafter predicts which masked position comes next

2 min
Pubblicato su Chiosa Aggiornato

Questa pagina non ha ancora una traduzione verificata. È mostrata la versione inglese. Leggi la versione inglese

In parole semplici

Diffusion language models fill in masked words a few positions at a time. To keep the model's original choices, a slower method fills one position per pass; filling several at once can change what it writes.

This paper introduces SPOT, which guesses the order in which positions will be filled, then asks the target model to check several possible continuations in one pass. On SDAR-8B tests, it used fewer target passes than a calibrated competing method, though verification can limit the speedup.

La ricerca

In block-diffusion decoding, a model chooses both what token to write and which masked position to fill. Exact decoding commits one position per pass, while faster multi-position methods can change the model's trajectory. Can a drafter propose several steps while preserving the target's own decoding choices?

SPOT learns which masked position a diffusion language model will fill next, then has the target verify a tree of candidate continuations in one pass.

SPOT predicts the next position from the target's most recent token probabilities, then builds candidate continuations for the target to check together. This lets it draft several possible steps while retaining a verification rule tied to the target's own choices.

In tests covering math and code tasks, the SDAR-8B target used fewer passes with SPOT than with Spiffy's calibrated draft graph, and SPOT had higher throughput. That measure counts generated tokens per generation-loop time relative to draft-free decoding on the same task, model and temperature.

The speed results come from specific models, tasks and decoding settings. Two transfer tests used an approximate prefix cache and therefore do not establish lossless decoding; verification costs can also erase speed gains, and at some settings on the smaller SDAR-1.7B model Spiffy was faster.

Analisi

Dettagli

Paper successivoBranching diffusion cuts the work of long text generation