Parallel language models coordinate several token predictions at once

Can a language model generate several tokens in one pass without losing the dependencies that make its answer coherent? Autoregressive models produce one token per pass, which can make inference slow as models scale. Parallel predictions may reduce that delay, but they need a way to coordinate.

Parallel Language Models train an autoregressive encoder and parallel decoder together, using auxiliary signals to coordinate several token predictions in one pass.

The authors post-trained a 1.3-billion-parameter PLM and compared it with a same-size autoregressive model. On GSM8K, a test of math questions scored by answer correctness, the PLM reached 32.6% versus 27.9%; it processed 5.2 versus 0.9 questions per second on one GPU at batch size one.

This shows that this PLM configuration did better on that math test while processing questions faster in the reported setup. The comparison changes both training and decoding, so it does not isolate the effect of parallel generation.

The GSM8K result comes from one post-training seed, and the authors selected checkpoints using GSM8K accuracy, so it is a bounded comparison rather than an independent final test. The paper also reports no 3B autoregressive counterpart.