Research collection
#anchor-rehearsal
Research tagged anchor-rehearsal: explanations, evidence, limitations and connections between papers.
2 results
A cache compressor rehearses likely answers before questions arrive
Attic-KV rehearses likely quoted answers and context anchors to improve cache compression before future questions are known.
Reusing a student's first answers can improve distillation
Semi-on-policy distillation reuses a student's starting answers and outperformed standard distillation on 14 of 17 tested pairs.