A graph monitor transfers hallucination detection across language models

Can attention-based monitors still detect hallucinations when a new language model places useful attention patterns in different layer and head positions, and supplies no labeled examples for training?

Existing internal-signal monitors often need target-specific labels, while matching coordinates across models is unreliable.

LEMON turns attention maps into symmetry-aware token graphs, letting a monitor trained on other models score hallucinations on an unseen model.

The authors trained on five language models and tested on the sixth. On RAGTruth, which labels hallucinated tokens in retrieval-grounded answers, ROC-AUC (0–100) measures how well scores rank hallucinated tokens above other tokens.

LEMON averaged 69.4 points across six models and three tasks, versus 59.8 for token-probability scoring.

Because each test model supplied no training labels, the higher mean supports transfer in this setting; it does not mean every model-task score improved.

The numeric comparison is for retrieval-grounded hallucination detection on six held-out models and three RAGTruth tasks.

The paper’s broader evaluation covers open-weight, instruction-tuned decoder-only models and hallucination and jailbreak monitoring; it does not establish performance on other behaviors or multimodal models.