#cross-head-mixing

Research tagged cross-head-mixing: explanations, evidence, limitations and connections between papers.

1 results

1. PaperNLP & Language

   Virtual Head Attention cuts cache while preserving model quality

   Small-model tests found higher average accuracy; at 1.7B, Virtual Head Attention cut cache use versus GQA-8.

   Added to Chiosa Oct 14, 2026