Random remasking lets diffusion models revise and improve sequences

Can a masked diffusion model recover from a bad token it has already revealed, and can theory describe its actual generation when its denoiser is imperfect? Standard decoding freezes revealed tokens, while earlier analyses discussed ideal denoisers. This leaves unclear what repeated correction does under the learned models used in practice.

The authors model inference as repeated remasking and denoising, derive a bound for imperfect denoisers, and use it to design adaptive R²D.

With standard-training checkpoints, more correction rounds raised R²D accuracy from 19.7% to 93.5% on Sudoku-Hard and from 86.0% to 98.5% on 3-SAT; ReMDM also improved but remained lower at reported budgets. With the same TCT-trained checkpoint and task budgets, R²D scored 97.5% versus 92.3% on Sudoku-Hard and 99.3% versus 96.5% on 3-SAT.

The matched comparison supports an added advantage from R²D’s inference procedure in these settings.

The method’s stopping rule checks whether its predicted sequence stays unchanged across repeated steps; it does not directly measure distance from the data distribution.

The reported results cover two logical tasks and one coding test in distinct model and training settings, so they do not establish performance across all masked diffusion models or task families.