On-policy distillation has no consistent edge in core tests

When a student model learns from its own generated answers, does that source itself explain claims that it forgets less or generalizes better?

Earlier comparisons changed other parts of training too, so they could not show whether the examples, learning objective, or optimization process drove the difference.

The study varies training-example source, KL direction, and learning rate in distillation to test which choices shape student outcomes.

The researchers compared teacher-generated with student-generated training examples from matched student checkpoints, while varying how next-word predictions were compared and the learning rate.

After 150 training steps, they measured task accuracy, the mean score change across seven other tests, and the fraction of parameters changing by less than 10^-6.

Across three reasoning tasks and three random seeds, student-generated examples gave no consistent advantage in these outcomes. KL direction tracked task accuracy more clearly, while learning rate had a stronger relationship with forgetting and update sparsity. This suggests example source alone did not explain the claimed benefits in this setup.

The results cover students with at most 1.5 billion parameters, short reasoning traces, and a fixed teacher. On-policy runs also used a different memory-saving setting, so the implementations did not match in every detail.