arXiv cs.LGPaper
The Sirens' Song: When Proximal Background Context Overshadows Distant Evidence
Long-context LLMs have a real problem: they ignore distant information not because it's far away, but because nearby noise is louder. LYRA's solution is mechanically sound and shows gains across standard benchmarks. For builders using long-context models for retrieval-heavy tasks (RAG, document QA, multi-hop reasoning), this is a concrete win. The practical question is whether the improvement justifies reranking or other post-processing, or if you just need to be smarter about what goes in the context window.