Shorter answers often stop early instead of removing repetition
Beyond Truncation? Entailment-Licensed Elision and the Limits of Semantic Preservation in LLM Decoding · arXiv:2611.08975
In plain words
Language models can make answers shorter simply by stopping sooner. This paper asks whether a method can instead remove repeated material while keeping more of the original answer than a same-length cut from its beginning.
Its ELIDE method hides some completed spans from the reader after checking whether text already shown supports them. Automatic checks looked favorable, but readers found many first-version omissions lost content or damaged the answer’s form.
A more restricted version reduced that harm, though fresh tests did not meet the paper’s rule for calling it rare.
The research
A shorter answer can save reading while still losing details a reader needs. The relevant question is whether removing material already conveyed can do better than simply cutting off the answer, and whether automatic checks can tell when an omission damages it. This paper’s results make reader review central to judging that trade-off.
The paper compares training-free answer-shortening methods with same-length truncation, then tests whether hiding supported spans can preserve content without changing generation.
On 24 held-out prompts per model, ELIDE v1 beat same-length truncation on an automatic measure of how well shortened answers retained the original answer’s sentence meanings. That suggested selective omissions could preserve more content than stopping at the same length.
But readers found that 59.4% of the v1 omissions they inspected on a larger development frame lost content or damaged form. The automatic measure therefore did not capture an important cost of hiding spans.
The main evaluations used greedy decoding at batch size one, and study readers rather than external annotators assessed omissions. Fresh tests support a lower harmful share for ELIDE v2, but its pooled estimate did not meet the paper’s pre-set rule for calling harm rare.
What the evidence shows
- The test is more than length
The paper introduces training-free methods that shorten instruction-tuned language-model answers without changing model weights. Its standard for content selection compares a shortened answer with the same-length prefix of the unmodified answer.
A method must preserve more of the original answer’s content than that prefix; otherwise, it has shown only that stopping early can shorten text.
- Passing checks can still mean stopping
The entropy-gated logits processor (EGP) changes stop-token scores during generation, and it met pre-set quality margins while reducing generated tokens. But an audit found all 8 changed decisions in one Mistral-7B audit selected a stop token, and all 192 sealed-test outputs were prefixes of the unmodified answers. The comparison shows that passing those quality checks did not mean EGP selected redundant content.
- Automatic coverage misses damage
Entailment-Licensed In-Decoding Elision (ELIDE) hides spans from the reader when an entailment check says text already shown supports them; the model still processes the hidden spans. On held-out prompts, it beat same-length truncation on an automatic coverage measure for all three models, with coverage within 0.007 of the original-answer-against-itself ceiling. But that measure did not capture whether each omission remained useful in context.
- A narrower rule reduced harm
Replaying both versions on the same development answers showed that restricting which spans could be withheld reduced the harmful share among read omissions from 59.4% to 5.0%. Because the revision followed readings of those answers, the replay does not establish the same reduction on new answers.
- Fresh tests leave a boundary
On two fresh test frames, readers judged 37 of 363 inspected v2 omissions harmful, a pooled share of 10.2%. Its uncertainty interval reached 13.7%, above the paper’s 10% cutoff for calling harm rare. The fresh tests support improvement over v1, but not the stronger claim that v2 rarely harms.
Deep
- Original paper authors
- Mahmoud Alyosify, Mirna Imbabi, Eman Elkhamisi
- Submitted to arXiv
- License
- Not specified by source