Attention statistics can be extracted without storing full matrices
In plain words
The paper builds a way to collect clues from a language model’s attention without saving a huge table of all attention weights. A separate, frozen Qwen3 model reads a context, question, and answer, then a trained classifier uses those clues to judge whether the answer is supported by the context.
In one reported setup, collecting statistics for five detectors added less relative time to a Qwen3-8B model’s forward pass on longer inputs than on shorter ones. The detector checks support from the supplied context, not general truth.
The research
A detector of unsupported answers may need to inspect a model different from the one that wrote them, because the writer may not expose its internal workings.
Can a separate model use attention clues to check whether an answer fits its supplied context, and can those clues be collected without storing a huge attention matrix?
The paper adds an attention-statistics method and Triton backend that avoids storing full attention matrices, then tests proxy detectors for contextual support.
Can a detector check whether an answer fits its supplied context when the model that wrote it cannot be inspected? The paper tests a separate frozen Qwen3 proxy, which reads the context, question, and answer and uses a trained classifier to predict whether the context supports it.
Across three datasets, the authors compare classifiers with and without attention features and compare proxy detectors with prompted judges. A 4B proxy exceeded the strongest evaluated judge in test F1 on two datasets, while a 0.6B proxy did so on the third. F1 balances caught unsupported answers against false alarms.
These results cover the tested datasets, Qwen3 proxies, judges, and implementation. The proxy assesses support by the supplied context, and two implemented statistics differ from their published definitions; those versions were not compared in detection.
Analysis
Deep
- Original paper authors
- Andrei Volodichev, Maksim Savichev, Nikita Sviridov, Artem Yuliy, Dmitry Simakov
- Submitted to arXiv
- License
- CC BY 4.0