A regression method distills generative models into fewer steps

Diffusion and flow generators can require tens to hundreds of sequential network evaluations to produce a sample. How can a pretrained generator be trained to use only a few steps while learning from the teacher’s behavior along its own path?

SMD distills a pretrained generator by matching its few-step predictions to a frozen teacher along the student’s own path.

The authors train a student on its own few-step rollout, matching its predictions to a frozen teacher’s predictions at intermediate points along each coarse step. On FLUX. 1-dev, a four-evaluation SMD student scored 0.715 on GenEval, which measures compositional text-image alignment, versus 0.675 for π-Flow; on DPG-Bench for dense-prompt alignment, it scored 84.8 versus 84.3.

These results show stronger scores on both reported prompt-alignment measures for this model and evaluation setting. The method also adds training-only components to represent within-step variation and repel the student from a weaker teacher prediction.

The FLUX.1-dev comparison covers those models, inference settings and prompt-alignment tests; the source does not establish that SMD and π-Flow used the same initialization or identical training procedures.

Some reported 3D and reference-conditioned audio–video evaluations are qualitative, so they do not establish quantitative performance in those settings.