DPR deep reading

Series · 1 posts

← Back to Paper Reading

  • DPR: Turn Open-Domain QA into Dense Passage Retrieval, but Do Not Treat the Dual Encoder as Production RAG

    Intermediate Retrieval, memory, and production RAG
    DPR deep reading: Part 1 , Note: Aug 27, 2026 , Paper: 2020 , NLP

    A source-grounded reading of Karpukhin et al., EMNLP 2020: a BERT dual encoder trained with question–passage pairs and in-batch negatives replaces BM25 over Wikipedia passages via MIPS. On NQ, top-20 retrieval is 78.4% vs BM25 59.1%; end-to-end Exact Match is 41.5. This is the retriever RAG uses, not a generation platform.

    Understand it in 90 seconds
    Problem
    Open-domain QA depends on efficient passage retrieval; in practice the first stage is almost always sparse TF-IDF or BM25. Sparse matching struggles with synonyms and paraphrases, and it cannot learn a task-specific space from question–passage pairs.
    Core insight
    Replace that first stage with two independent BERT-base encoders: a passage encoder embeds Wikipedia passages offline into 768-d vectors and builds a FAISS index; a question encoder embeds the query online and retrieves with maximum inner product search (MIPS). Training uses gold positives plus in-batch negatives (and BM25 hard negatives), without ORQA/REALM-style expensive extra pretraining or periodic index rebuilds.
    Strongest evidence
    Table 2 top-20 / top-100 retrieval accuracy—on NQ, Single DPR reaches 78.4% / 85.4% versus BM25 59.1% / 73.7% (about +19.3 points at top-20); the abstract states a 9%–19% absolute gain. Table 4 end-to-end Exact Match: DPR 41.5 on NQ, above ORQA 33.3 and REALMNews 40.4. Figure 1: DPR trained on only 1,000 examples already beats BM25.
    Main boundary
    Memory is the 20 Dec 2018 English Wikipedia dump split into about 21.015 million 100-word passages; evaluation is English open-domain / extractive QA; similarity is dual-encoder dot product without late interaction; this is not a production hybrid, not citation faithfulness, and not agentic search / read / final.
    Read the full deep dive

For speaking invitations, internal engineering sessions, or architecture exchange, see the topics and public work I can bring into the conversation.

Speaking & contact