A Maximum-Entropy-Inspired Parser
2000 · paper · Computational linguistics
Presents a statistical parser for Penn Treebank-style parse trees using a maximum-entropy-inspired model. It reaches 90.1% average precision and recall on sentences of 40 words or fewer, a 13% error reduction over the best earlier single-parser results.
Checked: PDF from aclanthology.org, fetched 5 Oct 2026. Page numbers are those of that PDF.
7 findings
Substance Conclusion prints "an average precision/recall of 91.1% on sentences of length ≤ 40"; the abstract, introduction, Figure 1 (Char00 LR 90.1, LP 90.1) and Section 5 ("90.1% on the test corpus") all give 90.1%.
- p. 7 Section 6 Conclusion, first sentence
sion/recall of 91.1% on sentences of length
- p. 1 Abstract
90.1% average precision/recall for sentences of
- p. 5 Figure 1, row Char00, ≤ 40 words
90.1 90.1 0.74 70.1 89.6
- p. 6 Section 5, text under Figure 2
achieves an average precision/recall of 90.1% on
Judge's note: Checked the page images for pages 1, 5, 6 and 7. Page 7 clearly prints 91.1%; the abstract and introduction print 90.1%; Figure 1 gives 90.1/90.1 for Char00 at ≤ 40 words; page 6 says 90.1% on the test corpus. The conclusion names the same standard sections and the same length cut-off, and pairs the number with the same 89.5% (≤ 100) and the same 13% error reduction, which reproduces with 90.1 (1.5/11.4 = 13.2%) and not with 91.1 (21.9%). No other test set or measure is named that could explain the difference.
Second check: another model agreed
Report a problem with this finding
Substance Section 5: "Including this information within a standard deleted-interpolation model causes a 0.6% decrease from the results using the less conventional model"; Figure 2 gives Standard Interpolation 88.2/88.3 against MaxEnt-Inspired 89.0/89.2, a difference of 0.85 in average precision/recall (0.8 precision, 0.9 recall).
- p. 7 Section 5, paragraph beginning 'Next we add the less obvious conditioning events'
dard deleted-interpolation model causes a 0.6%
- p. 6 Figure 2, rows Standard Interpolation and MaxEnt-Inspired
Standard Interpolation 88.2 88.3
- p. 6 Figure 2, row MaxEnt-Inspired
MaxEnt-Inspired 89.0 89.2
- p. 7 Section 5, paragraph on grandparent and left-sibling labels
causes a 0.6% decrease from the results using the less conven-tional model
Only the authors can say what is right here.
Judge's note: Read Figure 2 on the page 6 image and the sentence on the page 7 image. The neighbouring deltas in the same passage reproduce exactly from Figure 2 averages (Marked Coordination minus Explicit Pre-Term = 0.60, text 0.6%; MaxEnt-Inspired minus Marked Coordination = 0.45, text 0.45%), so the convention is absolute difference of averages. On that convention the Standard Interpolation gap is 0.85; against Marked Coordination it is 0.40. Neither gives 0.6, and rounding of one-decimal entries cannot close a 0.25 gap. The following sentence ("worse than not using this information at all") does agree with the table. Whether the text or a Figure 2 entry is wrong only the author can say.
Second check: another model agreed
Report a problem with this finding
Reference or label Reference 11: "DARROCH, J. N. AND RATCLIFF, D. Generalized iterative scaling for log-linear models. Annals of Mathematical Statistics 33 (1972), 1470-1480." The volume for 1972 is recorded elsewhere as 43, not 33.
- p. 8 References, entry 11
Annals of Mathematical Statistics 33 (1972),
- p. 8 References, entry 11
Annals of Mathematical Statistics 33 (1972)
- p. 8 Reference 11
Annals of Mathematical Statistics 33 (1972), 1470-1480.
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that Darroch and Ratcliff's paper appeared in Annals of Mathematical Statistics volume 43 (1972), pages 1470-1480, and that volume 33 of the journal is 1962. The page 8 image clearly prints "33 (1972)", so this is not an extraction error. The pipeline's Crossref lookup (cluster C025, doi:10.1214/aoms/1177692379) also records volume 43, which supports the finding, but the fact itself is not in the paper.
Second check: another model agreed
Report a problem with this finding
Wording The features are introduced as "a set of features f1 . . . fj" and Equation 4 runs "for 0 ≤ i ≤ j" up to gj(a,H), but the exponent of Equation 3 ends "+...+λm(a,H)fm(a,H)"; the upper index should be j.
- p. 3 Section 3, sentence before Equation 3
with a set of features f1 . . . fj
- p. 3 Equation 3 (exponent)
λm(a,H)fm(a,H)
Judge's note: Checked on the page 3 image: the sentence before Equation 3 prints f1...fj, the exponent of Equation 3 ends in λm fm, and Equation 4, described as a rewrite of Equation 3, uses j. No separate count m is defined here; m is used on page 2 for the Markov order and the number of left labels.
Second check: another model agreed
Report a problem with this finding
Wording "Note the changes to the last three terms in Equation 7": only the last two terms differ from Equation 6; the fourth term, p(t | l,lp,tp,lb)/p(t | l,lp,tp), is the same in both equations.
- p. 4 Section 3, sentence after Equation 7
Note the changes to the last three terms in
- p. 4 Equation 7
p(t | l, lp, tp, lb)
- p. 3 Equation 6
p(t | l, lp, tp, lb)
Judge's note: Compared Equation 6 (page 3) with Equation 7 (page 4) on the images. Both have a leading p(t | l) and five ratios. Ratios 1 to 3 are identical, including p(t | l,lp,tp,lb)/p(t | l,lp,tp). Only the last two changed: the lg and hp terms are now conditioned on l,lp,tp alone. The next sentence could be read as describing the three terms that share the l,lp,tp base, but the lb term was not changed.
Second check: another model agreed
Report a problem with this finding
Wording "based upon the features used in Equation 7, f1(t, l), f2(t, l, lp), f3(t, l, lp) . . . .": f3 is given the same arguments as f2; following the terms of Equation 7 it should be f3(t, l, lp, tp).
- p. 4 Section 3, paragraph 'We make one more point on the connection of Equation 7'
f1(t, l), f2(t, l, lp), f3(t, l, lp) . . . .
Judge's note: On the page 4 image the feature list prints f3 with three arguments (t, l, lp), identical to f2. The successive terms of Equation 7 condition on (l), (l,lp), (l,lp,tp), so the third feature should add tp. Read from the full-page image; the subscripts are small but the argument list is three items long.
Second check: another model agreed
Report a problem with this finding
Wording "For example, when we first guess the lexical head we can move from computing p(r | l, lp, h) to p(r | l, t, lp, h)": h is in both expressions and what is added is t, so the example appears to mean "when we first guess the pre-terminal" (or the head's pre-terminal).
- p. 6 Section 5, paragraph 'The second major reason'
expansion. For example, when we first guess
- p. 6 Section 5, same paragraph
the lexical head we can move from computing
Only the authors can say what is right here.
Judge's note: Read on the page 6 image. The paragraph's topic sentence is "why first guessing the pre-terminal makes so much difference", and the example adds t to the conditioning of the rule; the lexical head is already present in the first expression. The wording could be a dropped "'s pre-terminal", so only the author can say what was meant; hence a query rather than a correction.
Second check: another model agreed
Report a problem with this finding
12 minor (typos and the like; most checked by the judge only)
Wording Section 5 writes the rule-expansion probability as "p(r | l, lp, h) to p(r | l, t, lp, h)" with an undefined r, where the expansion is denoted e elsewhere (Equation 1, "p(e | l, t, h, H)").
- p. 6 Section 5, backing off paragraph
p(r | l, lp, h) to p(r | l, t, lp, h)
- p. 3 Equation 3 text
a set of features fl... fj
Judge's note: Checked on the page 6, page 1 and page 2 images: e(c) is the defined symbol for the expansion and r is not defined anywhere. In context r evidently stands for the tree-bank rule, so a reader is not misled; it is a small notational inconsistency. The second point in this cluster (f1...fj against λm fm in Equation 3) is the same defect as C003 and is covered there.
Second check: another model agreed
Report a problem with this finding
Wording Conclusion: "the previous three best parsers on this test [5,9,17] all perform within a percentage point of each other"; in Figure 1 (≤ 100 words) Coll99 is 88.1/88.3, Char97 86.7/86.6 and Ratna99 86.3/87.5, a spread of about 1.55 points in average precision/recall (1.15 between Coll99 and Char97 at ≤ 40 words).
- p. 7 Section 6 Conclusion
test [5,9,17] all perform within a percentage
- p. 5 Figure 1, ≤ 100 words block
Char97 86.7 86.6 1.20 59.9 83.2
- p. 5 Figure 1, ≤ 100 words block
Coll99 88.1 88.3 1.06 64.0 85.1
- p. 5 Figure 1, ≤ 100 words, rows Char97, Coll99, Ratna99
86.7 86.6 1.20 59.9 83.2
Judge's note: Read Figure 1 and the Section 5 text on the page 5 image and the conclusion on the page 7 image. The cluster holds two separate points. (1) The conclusion's "within a percentage point" is looser than the paper's own Figure 1, where Coll99 exceeds Char97 by 1.55 and Ratna99 by 1.30 on the ≤ 100 averages; this is an overstatement in wording, kept as a minor observation. (2) The claim that "Without these enhancements Char97 performs at the 86.6% level for sentences of length ≤ 40" contradicts Figure 1 is not upheld: the sentence explicitly describes Char97 with the clustering and unsupervised-learning enhancements removed, a different quantity from the 87.5 for the full system given two sentences earlier, and page 6 uses 86.6 consistently as that stripped test-corpus figure. Its coincidence with the ≤ 100 LP entry (86.6) is not evidence of an error.
Second check: another model agreed
Report a problem with this finding
Typo Abstract: "model for conditioning and smoothing that let us successfully to test and combine many different conditioning events" has a stray "to" ("let us successfully test").
- p. 1 Abstract
us successfully to test and combine many differ-
- p. 1 Abstract
that let us successfully to test and combine many different conditioning events
Judge's note: Read on the page 1 image; the wording is printed as quoted, so it is not an extraction artefact.
Second check: none (judge only)
Report a problem with this finding
Typo Section 2: "the possible expansions of a constituent are fixed in advanced by extracting a tree-bank grammar" should read "in advance".
- p. 2 Section 2, first paragraph on the page
in advanced by extracting a tree-bank grammar
- p. 2 Section 2, first paragraph
are fixed in advanced by extracting a tree-bank grammar
Judge's note: Read on the page 2 image, left column, first paragraph; printed as quoted.
Second check: none (judge only)
Report a problem with this finding
Typo Left-hand side of Equation 7 reads "p(t | l, lp, tp, b, lg, hp) ="; the fourth conditioning item should be lb, as in Equation 6 and on the right-hand side of Equation 7.
- p. 4 Equation 7, left-hand side
p(t | l, lp, tp, b, lg, hp) =
- p. 3 Equation 6, left-hand side
p(t | l, lp, tp, lb, lg, hp) =
Judge's note: On the page 4 image the left-hand side of Equation 7 prints a bare b between tp and lg, while Equation 6 on page 3 prints lb in the same position and the right-hand side of Equation 7 uses lb. Read from the full-page image only; no crop was available.
Second check: none (judge only)
Report a problem with this finding
Typo Misspellings: Section 5 "the parser labled Char97 in Figure 1 [5]" (labeled), and reference 9 "Ph.D. Disseration, 1999" (Dissertation).
- p. 5 Section 5, second paragraph
labled Char97 in Figure 1 [5]
- p. 5 Section 5
labled Char97
- p. 8 Reference 9
Ph.D. Disseration
- p. 6 Section 5
can be considerable sharpened
Judge's note: Both are printed as quoted on the page 5 and page 8 images; the paper spells "labeled" correctly elsewhere. The third item one reviewer bundled here, "considerable sharpened", is handled in C012.
Second check: none (judge only)
Report a problem with this finding
Typo Section 4: "Looking in particular at the precision and recall figures, the new parser's give us a 13% error reduction": possessive "parser's" with no noun, followed by a plural verb.
- p. 5 Section 4, last paragraph
the new parser's give us a 13% error reduction
Judge's note: Printed as quoted on the page 5 image, right column. Probably an elliptical "the new parser's [figures] give us"; as printed it reads as an error.
Second check: none (judge only)
Report a problem with this finding
Typo Section 5: "the probability of various rule expansions can be considerable sharpened" should read "considerably sharpened".
- p. 6 Section 5, paragraph 'The second major reason'
sions can be considerable sharpened
- p. 6 Section 5, p. 137, right column
can be considerable sharpened
Judge's note: Printed as quoted on the page 6 image, right column.
Second check: none (judge only)
Report a problem with this finding
Typo Section 5: "It turns out that usefulness of this process had already been discovered by Collins" is missing an article ("that the usefulness").
- p. 6 Section 5, paragraph 'Between the Old model and the Best model'
expansion. It turns out that usefulness of this
Judge's note: Printed as quoted on the page 6 image, bottom of the left column; no word is lost at the line break.
Second check: none (judge only)
Report a problem with this finding
Typo Section 5: "on the line labeled “MaxEnt-Inspired’." opens with a double quotation mark and closes with a single one.
- p. 7 Section 5, left column
labeled “MaxEnt-Inspired’.
Judge's note: Page 7 image, left column: the opening mark is a double quote and the closing mark is a single apostrophe. The other label in the same paragraph, “Standard Interpolation”, has matched double quotes.
Second check: none (judge only)
Report a problem with this finding
Typo Section 6: "Neither of these results were anticipated at the start of this research." uses a plural verb with "Neither".
- p. 7 Section 6, paragraph 2
Neither of these results were anticipated
Judge's note: The sentence is printed as quoted on the page 7 image. Strict agreement calls for "was", but the plural after "neither of" plus a plural noun is widely accepted usage, so either form could be legitimate.
Second check: none (judge only)
Report a problem with this finding
Typo Reference 9: "University of Pennsylvania, Ph.D. Disseration, 1999." misspells "Dissertation".
- p. 8 References, entry 9
of Pennsylvania, Ph.D. Disseration, 1999.
- p. 8 References, entry 9
Ph.D. Disseration, 1999
- p. 8 Reference 9
Ph.D. Disseration, 1999.
- p. 7 Section 5
labeled "MaxEnt-Inspired'.
Judge's note: The page 8 image prints "Disseration" in entry 9. One reviewer note in this cluster also mentions the mismatched quotation mark around “MaxEnt-Inspired’ on page 7; that is a separate defect, handled under C021.
Second check: none (judge only)