EN
Merlin helps long-context models weight useful segments
Multi-State Linear Attention with Generalized Second-Order Routing · arXiv:2610.37645
In plain words
Some language models split long text into chunks and keep a separate memory for each. When answering a question, they combine information from those chunks.
This paper introduces Merlin, a method that helps decide which chunks matter by checking both the average match to the question and how much that match varies among the chunk’s words.
In a small diagnostic, Merlin gave the chunk containing the answer evidence the most weight among the methods compared. The authors also reported higher average scores on long-context tasks for two tested model types, on examples no longer than 16K tokens.
The research
When a query needs a fact buried in a long context, what should decide which compressed segment contributes most? Position can favor recency, while average key-query match can miss a segment containing a few strongly matching keys. That leaves open whether measuring variation within a segment can route useful information better.
Merlin stores two summaries of each segment’s keys so each query can weight segment outputs using both average match and variation in match.
The authors trained multi-state Gated DeltaNet and Mamba-2 models with Merlin and compared them with Meanpool, which weights segments by average alignment.
LongBench tests long-context question answering, summarization, and related tasks; across eleven subtasks, average scores were higher with Merlin: 21.9 versus 19.3 for Gated DeltaNet, and 20.6 versus 18.4 for Mamba-2.
In a separate diagnostic, Merlin gave the answer-containing segment the most weight among the compared methods. Together, these findings support improved routing and task scores in the tested models.
The LongBench evaluation includes only samples with up to 16K tokens. The separate routing diagnostic covers 30 TriviaQA examples at 4K context and measures segment weight, not answer accuracy, so these results do not establish performance on longer contexts or other models.
What the evidence shows
- Which segments should contribute
Multi-state linear attention splits a long context into segments, stores a memory for each, and combines their outputs for a query. Merlin is a method for this setup that changes how the outputs are weighted.
The question is whether a segment’s position or average match to the query captures enough information to decide how much it should contribute.
- Why averages can miss evidence
Position-based weighting does not inspect what a segment contains. Mean-based weighting checks the average match between the query and the segment’s keys, but that average can hide a few strongly matching keys among many weak ones.
Merlin addresses this gap by using both average alignment and its variation within the segment to weight the segment’s output.
- A targeted routing check
In a diagnostic on 30 randomly selected TriviaQA retrieval examples at 4K context, the authors measured the weight assigned to the segment containing the answer evidence. Merlin assigned it more weight than position-only or mean-only aggregation.
This is evidence about where information is routed; the diagnostic does not measure whether the model produced the correct answer.
- Scores across long-context tasks
The authors also tested multi-state Gated DeltaNet and Mamba-2 models on LongBench, which tests long-context question answering, summarization, and related tasks. Across eleven subtasks and samples capped at 16K tokens, Merlin had a higher average score than Meanpool, which uses average alignment alone. The comparison supports better scores for these tested models under the reported conditions.
Deep
- Original paper authors
- Hee Seung Hwang, Sanghyuk Chun, Xindi Wu, Olga Russakovsky, Zhiwei Deng
- Submitted to arXiv
- License
- CC BY 4.0