A sentence-level method reduces targeted distraction in medical QA
EXCISE: Locating and Excising Distracting Sentences in Medical Question Answering · arXiv:2610.35558
In plain words
A medical question can include an irrelevant sentence that nudges a model toward the wrong answer, even when the sentence does not change which answer is correct. Excise is a method that checks which sentence most affects the model’s answer and whether that sentence seems out of place in the case.
In tests of three models, Excise improved accuracy when sentences targeted wrong answers. Accuracy fell slightly on clean questions, so the method did not help every case.
The research
Can a medical question-answering model ignore an irrelevant sentence that pulls it toward a wrong option without also losing useful evidence? The sentence that most affects an answer could be a key clinical detail, so influence alone does not show whether it should be blocked.
Excise finds the sentence that most shifts a medical answer, then blocks it only when its internal representation clashes with the case.
Can a model block an irrelevant sentence that steers its answer without also suppressing useful evidence? The authors tested sentence blocking in a four-model diagnostic: fixed attention masks harmed accuracy when the added sentence pointed to the correct option, while blocking that known sentence kept accuracy similar across conditions.
Excise measures each sentence’s effect on the answer and blocks the most influential one only if the model marks it as inconsistent with the case. On MedQA with Targeted Distractors, a test that adds answer-preserving sentences favoring wrong options, three models gained 16.7–22.8 accuracy points over no intervention; clean-question accuracy fell by 1.5–2.1 points.
The evidence comes from synthetic, answer-preserving distractors in multiple-choice medical question tests, using models whose internal signals can be inspected.
Excise does not correct false medical claims or establish clinical readiness; it also requires at least three narrative sentences and adds substantial inference cost.
What the evidence shows
- A targeted distraction problem
Excise is an inference-time method for medical question answering. It addresses a specific problem: a sentence can be irrelevant to the correct answer but still push a model toward a particular wrong option.
The authors created MedQA with Targeted Distractors, a test that adds answer-preserving sentences designed to favor wrong options. Nine open-weight models lost 19–32 percentage points of answer accuracy on those questions compared with the corresponding clean questions.
- Why locate the sentence
The authors tested whether a general intervention could suppress distraction without locating its source. In a four-model diagnostic, fixed attention-head masks raised accuracy when an added sentence favored a wrong option, but sharply lowered it when the sentence pointed to the correct answer.
Blocking the known added sentence instead produced similar accuracy in both conditions. The contrast shows why an intervention must distinguish distracting content from useful evidence in each question.
- A signal of poor fit
The authors found a signal that may help make that distinction. They compared the model’s internal representation of each sentence read alone with that of the rest of the case read alone.
On clean questions, the sentence that most affected the answer was rarely the sentence that fit least with the case. When a distractor had changed a correct answer to a wrong one, it was most discordant in 78–93% of cases across three tests.
- A selective intervention
Excise uses two checks: it measures how much blocking each sentence weakens the current answer, then asks whether the most influential sentence seems inconsistent with the rest of the case. Across three models, it raised accuracy by 16.7–22.8 points on MedQA with Targeted Distractors compared with no intervention, while reducing clean-question accuracy by 1.5–2.1 points. The result supports selective help on these tests, with a remaining cost on clean questions.
- What the evidence covers
With one additional off-case sentence, Excise scored 7.2–10.0 accuracy points above a version that chose sentences by poor fit alone. It also improved accuracy on MedDistractQA and BiasMedQA, two other medical question tests.
These results broaden the tested settings, but the distractors were synthetic and the method needs access to a model’s internal signals. The authors say it is not validated for clinical use.
Deep
- Original paper authors
- Yerim Hong, Sohyun Son, Heeseung Chung, Mujeen Sung
- Submitted to arXiv
- License
- arXiv non-exclusive