Reusing a student's first answers can improve distillation

When a language model learns from a stronger teacher, should it keep generating fresh answers for training? New answers reflect the changing student, but they may also drift away from teacher guidance.

The answer matters because the response source is a central choice in how the student receives training.

Semi-OPD reuses answers sampled from the starting student but recalculates its training signal, testing whether fixed examples can teach better than continually refreshed ones.

Semi-OPD keeps the training responses fixed, but continues to update the student using the teacher's probabilities and the student's current probabilities.

The authors compared it with OPD, which samples fresh responses from the changing student, while matching prompts, teacher, objective, and core training setup.

Across 17 pairs, Semi-OPD outperformed OPD on 14 and was comparable on three, using accuracy on four mathematical tests. Lower initial overlap between teacher and student tended to accompany a larger gain.

This supports an advantage for reusing initial responses in many tested pairs, with alignment related to how large that advantage was.

The comparison covers 17 teacher–student pairs and mathematical tests, so it does not establish that Semi-OPD is better for every pair or task.

It also depends on the starting student: the authors report that Semi-OPD loses its advantage when started from a base student, and they give no numerical threshold for how capable the initial student must be.