NLP & Language
Research on NLP & Language: explanations, evidence, limitations and connections between papers.
28 results
Read the evaluation protocol before trusting a benchmark score
Imagine seeing different accuracy scores for the same video model on the same benchmark. The model weights may be unchanged, yet the evaluator can choose different frames, process them differently, or score the response by a different rule. Across video questions, Taiwan knowledge questions, and dependency repairs, these studies show why a result needs its protocol beside it. A compact test can make a comparison less costly in its tested setting, while its task coverage still limits the claim. For a repair, success also depends on which checks the verifier runs.
Match reused work to changing inputs and the outcome you need
A sparse update, a robot action, and a caption choice all invite reuse of earlier work. Yet the next input may change what is useful, and a faster calculation or a higher benchmark score may miss the outcome the user needs. These studies suggest a bounded rule: identify what persists as the workload changes, then measure the intended outcome in that workload. Their evidence concerns computation time, task success, and candidate selection. It does not establish a reduction in stored memory.
Dedicated model leads on Indian language speech tests
A dedicated model leads on average in shared Indian-language speech tests, with exceptions on long read speech.
A speech model confines feature matching to protect English
A multilingual speech method confines feature matching to adapters, lowering word error rates in reported English and multilingual test groups.
A tokenizer reuses chat history to cut first-token delays
Delta Tokenizer reuses session text to cut first-token delay in tested long-context SGLang chat requests.
HybridAgent combines latent reasoning with tool use in text
HybridAgent combines latent reasoning with tool interaction; staged math tests show gains, while a proof bounds cross-turn memory claims.
JetSpec++ drafts ahead while the target checks current tokens
Across seven tests, JetSpec++ averaged 8.94 committed tokens per round versus DSpark’s 6.89 on Qwen3-8B at temperature zero and budget 16.
Sparse attention stays close to full attention at long context
JSSA selects context tokens directly, staying near full-cache language-modeling scores while speeding decoding in tested setups.
A Brazilian Portuguese model adds a tested safety step
A Brazilian Portuguese model adds safety training and refuses most harmful prompts in a 218-item evaluation.
Only two agent-training methods beat GRPO on average
A comparison of ten text-agent training methods finds average gains are limited and do not consistently outweigh training cost.
OpenTWBench tests Taiwan knowledge and exposes score effects
OpenTWBench tests Taiwan knowledge and shows how answer-reading rules can sharply change measured accuracy.
Long agent tasks gain verified progress feedback
Tev-Gym adds verified progress feedback to long agent tasks, improving full solves on an 18-task computing-and-math test.
A vision-language reward model learns from caption edits
A label-free vision-language scorer learns from caption edits; verifier-trained selection reduces absent object mentions in tested captions.
Virtual Head Attention cuts cache while preserving model quality
Small-model tests found higher average accuracy; at 1.7B, Virtual Head Attention cut cache use versus GQA-8.
Solo scores often miss which models help a team
Compositional reinforcement learning improved team performance across tested tasks and also helped in selected held-out settings.
A schema draft helps route questions across private data owners
Schema-based routing was not significantly less accurate than the best fixed protocol in ten tested settings.
Judge disagreement clarifies rubrics but not human alignment
Disagreement-guided rubric edits raised judge agreement on news summaries, with no detectable gain in expert match.
A new user turn helps redirect failed reasoning
UPTURN redirects failed reasoning from a learned point and improves average accuracy across four reasoners on six math tests.
POrchestra lets agents revise task setups mid-task
On simulated app tasks with updates, POrchestra had higher valid-scenario success than ReAct under the same execution budget.
Parallel draft correction raises decoding speed without sequential repair
A contextual corrector improves speculative-decoding throughput while keeping candidate scoring parallel in tested serving settings.
A crisis-support test shows recognition is not enough
A clinician-authored crisis-support test finds models often recognize danger without connecting users to human help.
One language model gains accuracy by spending more inference loops
Elastic Latent Recurrence keeps one-loop scores near matched baselines and raises ten-task averages with additional loops.
Choosing fewer, better tokens improves distillation
DIAL-OPD ranks response tokens by disagreement and probability, improving mean accuracy over full-token distillation in math tests.
Byte models miss results that need calculation
A byte-level model’s loss-based patch signal finds computed results that entropy misses on arithmetic text.