Dense Passage Retrieval for Open-Domain Question Answering
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020 · paper · AI and machine learning
Shows that passage retrieval for open-domain question answering can use learned dense representations alone, via a simple dual-encoder. It beats a strong BM25 system by 9–19% absolute in top-20 retrieval accuracy and yields new state-of-the-art end-to-end QA results.
Checked: PDF from arxiv.org, fetched 4 Oct 2026. Page numbers are those of that PDF.
3 findings
Substance Section 6.2 says DPR-based models beat the previous state of the art on four of five datasets "with 1% to 12% absolute differences in exact match accuracy", but the Table 4 margins over the best prior result are about 1.1 to 3.8 points.
- p. 8 Section 6.2, first paragraph
with 1% to 12% absolute differences in exact match
- p. 8 Table 4
46.8
- p. 8 Table 4
50.6
Only the authors can say what is right here.
Judge's note: Read Table 4 on the page 8 image. Best DPR-based result minus best first-block result: NQ 41.5-40.4 = 1.1; TriviaQA 57.9-56.0 = 1.9; WQ 42.4-40.7 = 1.7; TREC 50.6-46.8 = 3.8. The lower bound of about 1 matches but nothing reaches 12. Other readings do not give 1-12 either: excluding the concurrent REALM rows gives 7.0, 1.9, 6.0 and 20.5; against ORQA alone gives 8.2, 12.9, 6.0 and 20.5. No passage explains the range. It is probably left over from an earlier version of the table; the authors need to say what was intended.
Second check: another model agreed
Report a problem with this finding
Wording Table 2 caption describes the metric as "the percentage of top 20/100 retrieved passages that contain the answer", whereas Section 2 defines top-k retrieval accuracy as "the fraction of questions for which C_F contains a span that answers the question".
- p. 5 Table 2 caption
measured as the percentage of top 20/100 retrieved passages that contain the answer
- p. 2 Section 2, last paragraph
which is the fraction of ques-tions for which CF contains a span that answers the question
- p. 2 Section 2, last paragraph
which is the fraction of questions for which C_F contains a span that answers the question
Judge's note: Read the caption on the page 5 image and the definition on the page 2 image. Taken literally the caption describes a precision over passages, a different quantity from the per-question measure defined in Section 2 and used in Section 5.1 ("top-k accuracy"). No other passage redefines the metric. The caption is loosely worded; the evident fix is "percentage of questions for which the top 20/100 retrieved passages contain the answer".
Second check: another model agreed
Report a problem with this finding
Wording Section 6.2 says joint training "obtains a score of 39.8 EM" and that the pipeline is "outperforming a comparable joint training approach", while Appendix D says joint training results in "the same 39.8 exact match score on NQ dev as in our regular reader model training".
- p. 8 Section 6.2, joint-training paragraph
while outperforming a comparable joint train-
- p. 13 Appendix D, last sentence
resulting in the same 39.8 exact
- p. 8 Section 6.2, right column
obtains a score of 39.8 EM, which suggests that our strategy of training a strong retriever and reader in isolation can leverage effectively available supervision, while outperforming a comparable joint training approach
- p. 13 Appendix D, last sentence
resulting in the same 39.8 exact match score on NQ dev as in our regular reader model training
Only the authors can say what is right here.
Judge's note: Read Section 6.2 on the page 8 image and the end of Appendix D on the page 13 image. The main text does not name the split for 39.8; Appendix D says it is an NQ dev score and that the regular pipeline scores the same 39.8 there, which is a tie. "Outperforming" is therefore not supported by the appendix as written; the only apparent gap (41.5 in Table 4 vs 39.8) compares test with dev. The authors may have meant "matching with a simpler design".
Second check: another model agreed
Report a problem with this finding
7 minor (typos and the like; most checked by the judge only)
Typo Grammar slip in the Introduction: "Dense retrieval methods have thus never be shown to outperform TF-IDF/BM25" should read "never been shown".
- p. 1 Introduction, right column, last paragraph
Dense retrieval methods have thus never be shown to outperform
Judge's note: Read on the page 1 image, right column, last paragraph: the text prints "have thus never / be shown" across a line break. Not an extraction artefact.
Second check: none (judge only)
Report a problem with this finding
Typo Number disagreement in Section 3.1: "maps any text passage to a d-dimensional real-valued vectors" should read "vector".
- p. 2 Section 3.1, first sentence
maps any text passage to a d-dimensional real-valued vectors
Judge's note: Read on the page 2 image, Section 3.1, first sentence. The singular article with the plural noun is as printed.
Second check: none (judge only)
Report a problem with this finding
Typo Misspelling in the Table 2 caption: "using individial or combined training datasets" should read "individual".
- p. 5 Table 2 caption
using individial or combined training datasets
- p. 5 Table 2 caption
individial or combined training datasets
Judge's note: Visible on the page 5 image. The Table 4 caption on page 8 spells it "individual".
Second check: none (judge only)
Report a problem with this finding
Typo Misspelling in Section 5.2: "The middle bock is the in-batch negative training" should read "block".
- p. 6 Section 5.2, 'In-batch negative training', second paragraph
The middle bock is the in-batch negative training
Judge's note: Visible on the page 6 image, left column. The neighbouring sentences use "top block" and "bottom block".
Second check: none (judge only)
Report a problem with this finding
Typo Reference Lewis et al. 2020b prints an author's given name as "Aleksandara Piktus", apparently a misspelling of "Aleksandra".
- p. 10 References, Lewis et al. 2020b
Aleksandara Piktus
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that the cited paper's third author is named Aleksandra Piktus. The page 10 image prints "Aleksandara Piktus" as quoted; the correct spelling cannot be verified from the files.
Second check: none (judge only)
Report a problem with this finding
Typo Table 7 column header reads "Passage received by BM25"; it should read "retrieved", matching the adjacent "Passage retrieved by DPR".
- p. 13 Table 7, header row
Passage received by BM25
Judge's note: Visible on the page 13 image: the two headers differ, and the caption says "passages returned from BM25 and DPR". A slip for "retrieved".
Second check: none (judge only)
Report a problem with this finding
Typo Table 7 (second example, DPR passage) drops a non-ASCII letter in two places: "Pl Hagen was born" (the same cell prints "Pål Sverre Valheim Hagen") and "He lived in Jtten".
- p. 13 Table 7, second example, DPR passage
Pl Hagen was born
- p. 13 Table 7, second example, DPR passage
He lived in Jtten
Judge's note: Viewed both crops of page 13. The rendered image shows "Pl Hagen was born" and "He lived in Jtten", while the title and first line of the same cell print "Pål Sverre Hagen" / "Pål Sverre Valheim Hagen" with the å intact. So this is a character lost in the paper's typesetting, not an extraction artefact; the layout text agrees with the image. "Pl" for "Pål" is evident from the cell itself. That "Jtten" should read "Jåtten" rests on outside knowledge of the place name, though the missing-letter pattern is the same. Cosmetic only.
Second check: none (judge only)