NLP & Language
Research on NLP & Language: explanations, evidence, limitations and connections between papers.
6 results
Choosing fewer, better tokens improves distillation
DIAL-OPD ranks response tokens by disagreement and probability, improving mean accuracy over full-token distillation in math tests.
Byte models miss results that need calculation
A byte-level model’s loss-based patch signal finds computed results that entropy misses on arithmetic text.
A cache compressor rehearses likely answers before questions arrive
Attic-KV rehearses likely quoted answers and context anchors to improve cache compression before future questions are known.
TokenRouter helps models hand off requests without restarting work
TokenRouter coordinates model handoffs and raised output-token rates over existing systems in tested workloads.
Reusing a student's first answers can improve distillation
Semi-on-policy distillation reuses a student's starting answers and outperformed standard distillation on 14 of 17 tested pairs.
An optimizer change slows forgetting during adaptation
A training-time optimizer adjustment reduced old-domain validation-loss increases in seven stable Korean adaptation settings.