Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads

The paper introduces Logit-Contribution Scoring (LOCOS), a write-aware detector for attention heads that perform non-literal retrieval. Instead of rewarding heads whose attended token matches the generated token, LOCOS measures how each head’s OV-circuit output contributes to the answer-token unembedding direction, contrasting needle and off-needle positions in one forward pass. Across Qwen3, Gemma-3, and OLMo-3.1, ablating LOCOS-selected heads damages non-literal retrieval more efficiently than prior attention-based methods. On Qwen3-8B, ablating 50 heads reduces NoLiMa ROUGE-L from 0.401 to 0.000.