Research collection
#kl-divergence
Research tagged kl-divergence: explanations, evidence, limitations and connections between papers.
1 results
On-policy distillation has no consistent edge in core tests
A controlled distillation study finds no consistent on-policy edge, with gains on harder arithmetic not reliably persisting after further training.