Retrieval Systems: From Evidence to Production RAG

Series · 1 posts

← Back to Paper Reading

  • REVA: Moving RAG compression into reusable evidence views instead of paying per request

    Advanced
    Retrieval Systems: From Evidence to Production RAG: Part 1 , Note: Sep 15, 2026 , Paper: 2026 , NLP

    A critical reading of Nguyen et al.'s REVA (arXiv 2609.11209 v1): historical generator attention becomes a document-keyed score store, separating offline scoring from online rendering while exposing unseen-document fallback, local/global budgets, quality, and latency boundaries.

    Understand it in 90 seconds
    Problem
    A post-retrieval compressor that invokes another model, scores tokens, or rewrites text on every request can erase the latency benefit of a shorter context. A model-agnostic selector may also preserve information the target generator already knows while deleting evidence it actually needs.
    Core intuition
    Historical RAG requests already contain signals about how the generator used a document. Map attention from the query and available answer/response tokens onto readable word units, average those signals across repeated document accesses, and obtain a reusable evidence prior. This is a document-level retention tendency, not a query-specific answer.
    Strongest evidence
    With a fixed top-10 retrieval cache, four QA benchmarks, and three generators, Table I's full-split B=512 REVA-local result is higher than Trunc-local on NQ, TriviaQA, HotpotQA, and 2Wiki. Table II averages 37.83 F1, 26.98 EM, and 27.5 ms online overhead across 12 generator–dataset settings. Table III's all-seen 120-cell budget grid gives REVA-global 43.72 F1, 32.75 EM, and 49 ms.
    Main boundary
    All-seen keeps only held-out queries whose retrieved documents all have scores, so it is diagnostic rather than deployment coverage. Full-split includes prefix fallback. Attention is an evidence-importance proxy, not a citation verifier, and reported online overhead excludes score-store construction and updates.
    Read the full deep dive

For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.

Speaking & contact