Latent Dirichlet Allocation
2003 · paper · Computational linguistics
Describes latent Dirichlet allocation, a probabilistic model that represents each document in a collection as a mixture of underlying topics. The authors present approximate inference methods and report results in document modeling, text classification and collaborative filtering.
Checked: PDF from jmlr.org, fetched 5 Oct 2026. Page numbers are those of that PDF.
7 findings
Substance The A.4.2 Hessian is printed "∂L/∂αi∂αj = δ(i, j)MΨ′(αi)−Ψ′(∑k j=1 αj)", which does not follow from the gradient printed above it: differentiating gives M Ψ′(∑α) − δ(i,j) M Ψ′(αi), so the printed form has the opposite sign and lacks the factor M on the Ψ′(∑α) term.
- p. 30 Appendix A.4.2, Hessian equation
δ(i, j)MΨ′(αi)−Ψ′
- p. 30 Appendix A.4.2, gradient equation
Taking the derivative with respect to αi gives:
- p. 30 A.4.2 Dirichlet, Hessian equation
δ(i, j)MΨ′(αi)−Ψ′ ∑k j=1 α j
Only the authors can say what is right here.
Judge's note: Viewed page 30 image. The printed gradient is ∂L/∂α_i = M(Ψ(Σ_j α_j) − Ψ(α_i)) + Σ_d(Ψ(γ_di) − Ψ(Σ_j γ_dj)). My reworking: the second sum does not depend on α. d/dα_j of M·Ψ(Σα) is M·Ψ′(Σα) for every j; d/dα_j of −M·Ψ(α_i) is −δ(i,j)·M·Ψ′(α_i). So the Hessian is M·Ψ′(Σα) − δ(i,j)·M·Ψ′(α_i). The printed expression has both signs reversed and no M on the Ψ′(Σα) term (no parentheses group the two terms under M). The left side is written as the derivative of L itself, with no statement that it is negated. The summation index j inside Ψ′ also clashes with the free index j. The structural claim (diagonal plus constant, as in Eq. 10) still holds for the corrected form.
Second check: another model agreed
Report a problem with this finding
Reference or label Section 6 says "Using the algorithm in Section 5.1, we computed the variational posterior Dirichlet parameters", but the algorithm that computes γ and φ_n is in Section 5.2 (Variational inference).
- p. 15 Section 6, third paragraph
Using the algorithm in Section 5.1, we computed the variational posterior Dirichlet
- p. 27 Appendix A.3, first line
described in Section 5.1. Recall that
Judge's note: Page 15 image shows the sentence as quoted. The text layer has headings "5.1 Inference" and "5.2 Variational inference"; the variational algorithm is under 5.2, and Section 7.3 (page 22 image) cites "the variational inference method described in Section 5.2" for the same method. This cluster is taken as the Section 6 instance; the Appendix A.3 instance is judged in C015.
Second check: another model agreed
Report a problem with this finding
Reference or label Appendix A.3 says "we derive the variational inference algorithm described in Section 5.1", but that algorithm (the variational distribution and its updates) is described in Section 5.2 (Variational inference).
- p. 27 Appendix A.3, first sentence
we derive the variational inference algorithm described in Section 5.1
- p. 22 Section 7.3
variational inference method described in Section 5.2
- p. 15 Section 6, third paragraph
Using the algorithm in Section 5.1, we computed the variational posterior Dirichlet
- p. 27 A.3 Variational inference
variational inference algorithm described in Section 5.1
Judge's note: Page 27 image shows the sentence as quoted. The text-layer headings are "5.1 Inference" and "5.2 Variational inference", and Section 7.3 (page 22 image) cites Section 5.2 for the same method. This is the Appendix A.3 instance of the wrong pointer; the Section 6 instance is a separate location and is confirmed under C001, so the two are not merged as duplicates.
Second check: another model agreed
Report a problem with this finding
Reference or label Reference entry "C. Papadimitriou, H. Tamaki, P. Raghavan, and S. Vempala. Latent semantic indexing: A probabilistic analysis. pages 159–168, 1998." gives no venue.
- p. 25 References
Latent semantic indexing: A probabilistic analysis. pages 159–168, 1998.
Only the authors can say what is right here.
Judge's note: The page 25 image shows the entry with a title, page range and year but no proceedings or journal name. The omission is in the published page, not lost in extraction. Only the authors can supply the venue.
Second check: another model agreed
Report a problem with this finding
Reference or label Appendix A.3 says "we derive the variational inference algorithm described in Section 5.1", but the variational inference algorithm is in Section 5.2 ("Variational inference"); Section 5.1 is "Inference".
- p. 27 Appendix A.3, first sentence
we derive the variational inference algorithm described in Section 5.1
Judge's note: Viewed page 27 image: the sentence reads as quoted. The text layer shows the headings "5.1 Inference" and "5.2 Variational inference"; a later passage in the article itself refers to "the variational inference method described in Section 5.2". The pointer in A.3 should be to Section 5.2.
Second check: another model agreed
Report a problem with this finding
Wording "The latent variable models perform better than the simple unigram model." is unqualified, but in Figure 9 the fold-in pLSI curve lies above (worse than) the smoothed unigram line at the smallest numbers of topics in both panels.
- p. 20 Section 7.1, last paragraph
The latent variable models perform better than the simple unigram model.
- p. 18 Figure 9, top and bottom panels
Fold in pLSI
Judge's note: Figure 9 on the page 18 image. Top panel: pLSI is at about 3200 and 2980 for the two smallest k, against a unigram line near 2890. Bottom panel: pLSI is at about 6700, 6500 and 6200 for the three smallest k, against a unigram line near 5800, crossing below near k≈20. LDA and the smoothed mixture of unigrams are below the unigram line at every plotted point, so one reviewer's side remark about the mixture of unigrams is not supported. The sentence reads as a general trend that overstates the plotted data for pLSI at small k.
Second check: another model agreed
Report a problem with this finding
Wording Appendix A.3: "as a surrogate for the posterior distribution p(θ,z,w|α,β)" writes the joint distribution; the posterior is p(θ,z|w,α,β), as in Eq. (13).
- p. 27 Appendix A.3, sentence after Eq. (11)
as a surrogate for the posterior distribution p(θ,z,w|α,β)
- p. 27 Appendix A.3, Eq. (13)
D(q(θ,z|γ,φ) ∥p(θ,z|w,α,β))
Judge's note: Page 27 image shows w printed before the conditioning bar in the sentence after Eq. (11). Eq. (13) on the same page writes the true posterior as p(θ,z|w,α,β), and Eq. (12) uses p(θ,z,w|α,β) for the joint. The word "posterior" makes the intended object and the fix evident. The slip is notational and does not affect the derivation.
Second check: another model agreed
Report a problem with this finding
10 minor (typos and the like; most checked by the judge only)
Wording Text says for pLSI "As k gets larger ... the perplexity grows", but in Table 1 the pLSI perplexity falls from 1.72×10^7 at k=100 to 1.31×10^7 at k=200.
- p. 20 Section 7.1, first paragraph on the page
decreases and thus the perplexity grows
- p. 19 Table 1, pLSI column, rows k=100 and k=200
1.31×107
Judge's note: Table 1 on the page 19 image: the pLSI column rises from 7,052 at k=2 to 1.72×10^7 at k=100, then drops to 1.31×10^7 at k=200. The sentence on page 20 describes a general tendency that holds for k=2 to 100; the reversal in the last row is not mentioned. A minor consistency point only.
Second check: another model agreed
Report a problem with this finding
Wording "Note that we reduce the feature space by 99.6 percent in this case": going from 15,818 word features to 50 LDA features is a 99.68% reduction, which rounds to 99.7 percent.
- p. 20 Section 7.2, last paragraph
reduce the feature space by 99.6 percent in this case
- p. 20 Section 7.2
dataset contains 8000 documents and 15,818 words
- p. 20 Section 7.2
The dataset contains 8000 documents and 15,818 words.
- p. 20 Section 7.2, last paragraph
Note that we reduce the feature space by 99.6 percent in this case.
Judge's note: Page 20 image shows "8000 documents and 15,818 words", "50-topic LDA model" and "99.6 percent". The recomputed reduction is 99.684%, so 99.6 is reached only by truncation, or if the SVM used a slightly different word-feature count that the paper does not state. A 0.1-point difference where truncation is plausible, so either form could be legitimate.
Second check: another model agreed
Report a problem with this finding
Wording Appendix A.2: "In general, this algorithm scales as O(N3) due to the matrix inversion" uses N for the problem dimension, although N is the document length elsewhere and the matrix here is k×k.
- p. 26 Appendix A.2
this algorithm scales as O(N3) due to the matrix inversion
Judge's note: Page 26 image shows "O(N^3)". The sentence begins "In general", so N can be read as a generic dimension for Newton-Raphson. But the same appendix sums j=1..k and refers to "2k values" (page 27), and N is used for the number of words in a document elsewhere. A notational inconsistency; the generic reading is legitimate.
Second check: another model agreed
Report a problem with this finding
Typo Misspelling in Section 7.1: "is algebraicly equivalent to the inverse of the geometric mean per-word likelihood" should read "algebraically".
- p. 16 Section 7.1, first paragraph
is algebraicly equivalent to the inverse
- p. 16 Section 7.1, first paragraph
algebraicly equivalent
Judge's note: The misspelling is visible on the page 16 image, mid-line, so it is not a line-break or extraction artefact.
Second check: none (judge only)
Report a problem with this finding
Typo The predictive-perplexity formula in Section 7.3 has an unmatched closing parenthesis after the fraction, before the closing brace: "exp{ −Σ log p(w_{d,N_d} | w_{d,1:N_d−1}) / M ) }".
- p. 22 Section 7.3, predictive-perplexity equation
M )
Judge's note: The stray ")" after the fraction with denominator M is visible on the page 22 image, with no matching opening parenthesis.
Second check: none (judge only)
Report a problem with this finding
Typo Reference Avery (2002) prints "Caenorrhabditis genetic center bibliography"; the genus is spelled "Caenorhabditis".
- p. 24 References, Avery (2002)
Caenorrhabditis genetic center bibliography
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: the correct spelling of the genus name is Caenorhabditis, with a single r. The page 24 image shows "Caenorrhabditis" with a double r. The paper spells the genus out nowhere else (it uses "C. Elegans"), so there is no internal comparison.
Second check: none (judge only)
Report a problem with this finding
Typo Reference Ronning (1989) prints "Journal of Statistcal Computation and Simulation"; "Statistcal" should be "Statistical".
- p. 25 References, Ronning (1989)
Journal of Statistcal Computation and Simulation
- p. 25 References, Ronning (1989)
Journal of Statistcal Com-
- p. 25
Journal of Statistcal Com- putation and Simulation, 34(4):215–221, 1989.
Judge's note: The page 25 image shows "Journal of Statistcal Com-putation and Simulation" with the i missing. A plain misspelling.
Second check: none (judge only)
Report a problem with this finding
Typo Heading of Appendix A.1 reads "Computing E[log(θi |α)]", with the conditioning bar inside the logarithm, while the body derives "E[logθi |α] = Ψ(αi)−Ψ(Σαj)".
- p. 26 Appendix A.1 heading
Computing E[log(θi |α)]
- p. 26 Appendix A.1, result
E[logθi |α] = Ψ(αi)−Ψ
Judge's note: Page 26 image: the heading has the parentheses enclosing "θ_i | α" inside the log; the displayed result has the conditioning outside the log. Cosmetic notation inconsistency; the meaning is clear from the body.
Second check: none (judge only)
Report a problem with this finding
Typo In A.3.1 the Lagrangian constraint term is printed "λn (∑k j=1 φni −1)" (sum over j, summand indexed by i; should be φnj), and the multiplier λn appears as a bare λ in the derivative "+logβiv −logφni −1+λ."
- p. 28 Appendix A.3.1, Lagrangian L[φni]
λn (∑k j=1 φni −1)
- p. 28 Appendix A.3.1, derivative
+logβiv −logφni −1+λ.
Judge's note: Viewed page 28 image. The Lagrangian term is printed with the sum over j=1..k and summand φ_ni, while the constraint stated just above is Σ_{i=1}^k φ_ni = 1. The derivative line ends in "+ λ" with no subscript n. Both are index slips; the meaning is recoverable and Eq. (16) is unaffected.
Second check: none (judge only)
Report a problem with this finding
Typo In A.3.2, L[γ] mixes summed and free uses of index i: the terms "∑N n=1 φni (Ψ(γi)−Ψ(∑γj))" and "+logΓ(γi)" have i free while the other terms carry ∑k i=1; Eq. (15) has ∑i on both.
- p. 29 Appendix A.3.2, first and second L[γ] equations
−logΓ (∑k j=1 γj) +logΓ(γi)
Judge's note: Viewed page 29 image. In the first L[γ] expression the first and last terms have Σ_{i=1}^k, but the φ_ni term has only Σ_{n=1}^N and logΓ(γ_i) has no sum over i. The simplified form likewise ends in "+ logΓ(γ_i)" outside the sum. Eq. (15) on page 28 has Σ_i on the corresponding terms. This is a notational inconsistency only; the derivative and Eq. (17) that follow are consistent with the fully summed form.
Second check: none (judge only)