A sentence-level reward helps retrieval models ground their own writing
Copying Is Not Grounding: Training Faithful Long-Form RAG with Authored-Sentence Credit · arXiv:2610.18442
In plain words
Some AI systems answer questions using passages retrieved from documents. A checker can mark copied text as supported, so training to maximize supported sentences may reward copying instead of teaching a system to support its own writing.
Authored-Sentence Credit (ASC) is a training method that scores sentences one at a time. Copied sentences and statements supported by the question alone earn no credit; new sentences must be supported by the passages.
Across the tested models, a separate evaluator found a median 38% relative reduction in the share of model-written sentences it judged unsupported after reading their passages, while whole-answer judges rated answer quality on par with the base models.
The research
When a model answers from retrieved passages, what counts as evidence that it can support its own claims? A sentence copied from a passage may be correct without showing the model can ground what it writes.
The paper asks whether training credit can distinguish passage copying from evidence-backed authorship.
Authored-Sentence Credit gives each sentence its own training score, rewarding passage-supported writing while withholding credit from copied or evidence-free text.
Four instruction models trained on two question-answering corpora were compared using shared settings.
Across 20 model-corpus settings, a separate Gemma-4-31B judge found a median relative reduction of 38% versus corresponding base models in the share of authored sentences it judged unsupported after reading their passages.
Whole-answer judges rated quality on par with base models, supporting the authors’ view that grounded writing should earn the reward.
The evidence covers four models trained on two corpora, and each baseline had one training run. The paper does not specify whether each trained model began from the same checkpoint as its base-model comparator.
What the evidence shows
- The reward can favor copying
Authored-Sentence Credit (ASC) is a training method for long-form retrieval-augmented generation (RAG), where models answer questions using passages retrieved for each question. It addresses a mismatch: the usual faithfulness reward counts the share of answer sentences a checker marks supported by passages, but copied sentences can pass that check. A higher score therefore need not mean the model’s own writing has become more grounded.
- Human labels expose the gap
To check whether this mismatch appears beyond the automatic reward checker, human span annotators labeled sentences in RAGTruth responses. They marked 12.2% of authored sentences erroneous, compared with 1.9% of copied sentences, across 131,087 labeled sentences.
The researchers calculated each percentage within its own provenance group. This comparison applies to the labeled RAGTruth responses, not to all RAG answers.
- How sentence credit works
ASC addresses the mismatch by giving credit only when a sentence is authored, not supported by the question alone, and supported by the retrieved passages. It compares each sentence’s credit with credits for sentences in answers sampled for that same question.
Thus copied, question-supported, or unsupported sentences rank at the bottom. If any sentence earns credit, those failing a test get a negative training weight; if none does, they still receive no positive reinforcement.
- The main results measure support
Researchers trained four instruction models on two question-answering corpora and compared ASC with three other training signals under shared settings. Across 20 model-corpus settings, a separate Gemma-4-31B judge found a median relative reduction of 38% versus corresponding base models in the share of authored sentences it judged unsupported after reading their passages.
A second evaluator, MiniCheck-Flan-T5-Large, found a median 46% increase in supported authored sentences per answer, a count rather than a share.
- The gains did not trigger shortcuts
Those error improvements need to be checked against shortcuts such as copying, empty claims, refusals, or shorter answers. Copy is the fraction of answer words covered by verbatim five-word matches with passages; Vac. is the share of passage-supported authored sentences that the question alone also supports.
None of ASC’s 20 settings was flagged: a setting counted as a shortcut if Copy, Vac., or refusal rate exceeded the corresponding base model’s by more than 3 percentage points, or answer length fell below 85% of its. Two judges compared whole answers for how well they served a reader; their mean win rates for ASC versus base models were 51.0% and 49.1% across 20 settings, with ties and disagreements counted as half wins.
- Transfer and control results
On ClapNQ, ELI5, and ExpertQA, three question-answering tests that were entirely absent from training and use retrieved passages, the median reduction in authored error was 38% across four models versus their base models.
In a separate Qwen3.5-9B ablation, replacing sentence-level comparison with a whole-answer score raised authored error on four of five corpora. This supports the role of sentence-level credit in that tested setup and bounds the transfer result to those three corpora.
Deep
- Original paper authors
- Hoang Pham, Duc Anh Vu, Duy Tung Doan, Cong-Duy T Nguyen, Anh Tuan Luu
- Submitted to arXiv
- License
- CC BY 4.0