Random Forests

• p. 2 Section 1.1, first paragraph

  a random selection (without replacement) is made from the examples in the training set

• p. 8 Section 3.1, first paragraph

  Each new training set is drawn, with replacement, from the original training set

• p. 2 Section 1.1 Introduction, first paragraph

  a random selection (without replacement) is

• p. 8 Section 3.1, first paragraph

  Each new training set is drawn, with replacement,

Judge's note: Read pages 2 and 8 images. Page 2 prints "(without replacement)"; the same page describes bagging as counts from N darts thrown at N boxes (i.e. with replacement), and page 8 says "drawn, with replacement" and "bootstrap training sets" with about one-third left out. No statement in the paper explains a without-replacement variant. The intro parenthetical contradicts the rest of the paper.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 4 Theorem 1.2 and equation (1)

  PE* converges to

• p. 27 Appendix I, proof of Theorem 1.2, first sentence

  It suffices to show that there is a set of probability zero C on the sequence space

Only the authors can say what is right here.

Judge's note: Read page 4 (theorem) and page 27 (proof) images. The proof shows vote fractions converge for all x off a null set, so mg→mr pointwise; but PE*=P(mg<0) integrates an indicator that is discontinuous at 0, so the limit follows only where mr≠0. Counterexample reworked: two classes, P(Y=1)=p, h independent of x and equal to each class with probability 1/2. Then mr≡0 and (1)=P(0<0)=0. With f_N the fraction of the first N trees voting class 1, PE*_N = p·1(f_N<1/2)+(1−p)·1(f_N>1/2); N(f_N−1/2) is a symmetric random walk, a.s. on each side of 0 infinitely often, so PE*_N takes the values p and 1−p infinitely often and does not converge to 0. Nothing in Section 2.1 or Appendix I assumes P(mr=0)=0. The theorem needs that extra hypothesis (or a liminf/limsup statement); it is a boundary case and the authors should say which.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 28 Appendix II, equation (A2)

  sd(Θ)=[p1+ p2+( p1−p2)2]

• p. 5 Definition 2.2

  rmg(Θ,X,Y)= I(h(X,Θ)=Y)−I(h(X,Θ)= ˆj(X,Y))

Only the authors can say what is right here.

Judge's note: Page 28 image shows a plus sign before (p1−p2)^2. Reworking from Definition 2.2: rmg = I(h=Y) − I(h=ĵ) with ĵ≠Y, so the two events are disjoint and rmg takes +1 with probability p1, −1 with probability p2, and 0 otherwise. Then E[rmg^2] = p1+p2 and E[rmg] = p1−p2, so var = p1+p2−(p1−p2)^2. Section 2 defines sd(Θ) as the standard deviation of rmg holding Θ fixed. Counterexample: p1=1, p2=0 gives rmg≡1 and true sd 0, while the printed formula gives sqrt(2). With p1=0.75, p2=0.25 the printed variance is 1.25, above the maximum of 1 for a variable in [−1,1]. The formula is wrong as printed.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 6 Section 2.2, two-class simplification

  ρ =EΘ,Θ' [ρ(h(⋅,Θ),h(⋅,Θ')]

Only the authors can say what is right here.

Judge's note: Read page 6 image. With Y,h∈{±1}, rmg=Y·h, and corr(Yh,Yh') is not corr(h,h') in general; also ρ̄ is defined at the top of the page as an sd(Θ)sd(Θ')-weighted average, not a plain expectation of correlations. Counterexample reworked: X uniform on 4 points, Y=(1,1,−1,−1), h1=(1,1,−1,1), h2=(1,−1,−1,−1), each with probability 1/2. Raw margins Yh1=(1,1,1,−1), Yh2=(1,−1,1,1): means 0.5, E[product]=0, cov=−0.25, variances 0.75, corr=−1/3. Classifiers: means 0.5 and −0.5, E[h1h2]=0, cov=+0.25, variances 0.75, corr=+1/3. The sds are equal, so ρ̄=(1+1−1/3−1/3)/4=1/3, whereas E ρ(h,h')=(1+1+1/3+1/3)/4=2/3. Even if ρ(h,h') is read as the correlation of the correctness indicators, the equality with the weighted ρ̄ needs equal sd(Θ) across Θ. Theorem 2.3 is unaffected; the displayed interpretive formula is false as printed.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 11 Section 4, last paragraph

  For zip-code data, using F=1, the ratio is .025, implying that Forest-RI is 40 times faster.

• p. 11 Section 4, compute-time paragraph

  using F=1, the ratio is .025, implying that Forest-RI is 40 times faster

• p. 9 Table 1, zip-code row

  7291

• p. 11 Section 4

  the ratio is .025, implying that Forest-RI is 40 times faster

Only the authors can say what is right here.

Judge's note: Viewed p11 and Table 1 on p9 (zip-code: train 7291, inputs 256). log2(7291) is about 12.83, since 2^13 = 8192; 12.83/256 = 0.050 and 1/0.050 = 20. The printed .025 would need log2(N) of about 6.4. Natural log (8.89/256 = 0.035) does not give it either. No other N or definition is given in the text. The empirical figure (4 minutes vs almost three hours) is a comparison with Adaboost, not the formula. Only the authors can say whether the formula or the value is intended.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 19 Section 10, paragraph after Figure 4

  The reason that variable 6 seems important, yet is no help once variable 2 is entered

• p. 19 Section 10, preceding paragraph

  But when variable 8 is added, the error falls only to 29.4%. When variable 6 is added to variable 2, the error falls to 26.4%.

Judge's note: Page 19 image: "But when variable 8 is added, the error falls only to 29.4%. When variable 6 is added to variable 2, the error falls to 26.4%." The same paragraph ends "the 8th variable carries some of the same information as the second. So it does not add predictive accuracy when combined with the second." The paragraph contradicts itself and the fix is evident.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 25 Table 8, Ozone row, With Bagging

  17.8

• p. 23 Table 6, Ozone row, Forest

  16.3

• p. 24 Table 7, Ozone row, Test Error

  16.3

• p. 25 Table 8

  Ozone 17.8 16.3

Only the authors can say what is right here.

Judge's note: Page images: Table 6 Forest column is 10.2, 16.3, 24.6, 4.6, 4.2, 5.7, 19.6, 21.6. Table 7 Test Error is the same, with Ozone 16.3. Table 8 With Bagging is 10.2, 17.8, 24.6, 4.6, 4.2, 5.7, 19.6, 21.6. The value 17.8 equals the Bagging and Adapt. Bag entries for Ozone in Table 6, and Table 8's With Noise entry for Ozone is 16.3. The text describes the same setup (25 features, 100 trees) and nothing explains a different run. Which Ozone value is right for each column is for the author to say; it bears on the comparison of noise with bagging for that data set.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 2 Section 1.1, second paragraph

  Breiman [1999] generates new training sets by randomizing the outputs in the original training set.

• p. 26 References

  Breiman, L. [1999] Using adaptive bagging to debias regressions

• p. 2 Section 1.1, second paragraph

  Breiman [1999] generates new training sets by randomizing the outputs

• p. 26 References

  Breiman. L. [1998b] Randomizing Outputs To Increase Prediction Accuracy.

Judge's note: Read pages 2 and 26 images. The only 1999 Breiman entry in the reference list is the adaptive bagging report; "Randomizing Outputs To Increase Prediction Accuracy" is [1998b]. The text of Section 12 cites "outputs (Breiman [1998b])" for the same method, and [1999] is used for adaptive bagging elsewhere (text layer). Verified entirely within the paper's own reference list.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 13 Section 5.1, fourth paragraph

  Breiman et al [1985]

• p. 23 Section 12, paragraph after Table 5

  They originated in Friedman [1991] and are also described in Breiman [1998]

• p. 2 1.1

  (Breiman [1996])

• p. 23 Section 12

  described in Breiman [1998]

Only the authors can say what is right here.

Judge's note: Read the reference list (pp.26-27) in the text layer: the Breiman entries are 2000, 1999, 1998a, 1998b, 1996a and 1996b only. There is no Breiman et al entry, and the unsuffixed citations cannot be resolved to one entry. The citing text was checked on the page 23 and page 10 images; the page 13 and page 2 citations were checked in the text layer only. The missing Friedman [1991] entry is handled under C045.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 3 Section 1.2 Outline

  These are called out-of-bag estimates and are reviewed in Section 4.

• p. 8 Section heading

  3.1 Using Out-Of-Bag Estimates to Monitor Error, Strength, and Correlation

• p. 3 Section 1.2 Outline, second paragraph

  estimates and are reviewed in Section 4

• p. 3 1.2 Outline

  reviewed in Section 4

Judge's note: Read page 3 image (outline) and page 8 image (Section 3.1 heading and out-of-bag discussion). The Section 4 heading was confirmed in the text layer. Section 6 text itself refers to "as described in Section 3.1". The outline pointer is off by one.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 3 Section 1.2 Outline

  Sections 5 and 6 give empirical results for two different forms of random features.

• p. 9 Section heading

  4.Random Forests Using Random Input Selection

• p. 11 Section heading

  5. Random Forests Using Linear Combinations of Inputs

• p. 3 Section 1.2 Outline, second paragraph

  Sections 5 and 6 give empirical results for two different forms of random features

Judge's note: Page 3 image shows the outline sentence. Section headings "4.Random Forests Using Random Input Selection", "5. Random Forests Using Linear Combinations of Inputs" and "6. Empirical Results on Strength and Correlation" were confirmed in the text layer (heading page images not opened). The pointer is off by one.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 3 Section 1.2 Outline

  an empirical study is carried out in Section 7

• p. 13 Section heading

  6. Empirical Results on Strength and Correlation

• p. 3 Section 1.2 Outline, third paragraph

  carried out in Section 7

Judge's note: Page 3 image shows "carried out in Section 7". Headings for Sections 6 and 7 were confirmed in the text layer (heading page images not opened). The pointer is off by one.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 3 Section 1.2 Outline

  Evidence for this conjecture is given in Section 8.

• p. 15 Section heading

  7. Conjecture: Adaboost is a Random Forest

• p. 3 Section 1.2 Outline, fourth paragraph

  conjecture is given in Section 8

• p. 16 Section 8 heading

  8. The Effects of Output Noise

Judge's note: Page 3 image shows "conjecture is given in Section 8". Headings for Sections 7 and 8 were confirmed in the text layer (heading page images not opened). This is a distinct pointer from C005–C007, though all share the same stale-numbering cause; the outline pointers from Section 9 onward agree with the headings.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 7 Section 3, first paragraph

  (Breiman [1998c])

• p. 7 Section 3, first paragraph

  the outputs (Breiman [1998c]) also does better

Judge's note: Page 7 image shows "(Breiman [1998c])". Page 26 image shows only [1998a] Arcing Classifiers and [1998b] Randomizing Outputs; a text search finds no 1998c entry. This is a dangling citation, and the content matches [1998b]. The same paragraph's "see Breiman [1998b]" for arcing appears correspondingly shifted (the arcing paper is [1998a]), which suggests stale suffixes.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 7 Section 3, first paragraph

  adaptive reweighting (arcing) of the training set (see Breiman [1998b],

• p. 26 References

  Breiman, L. [1998a], Arcing Classifiers

• p. 7 Section 3, first paragraph

  work by adaptive reweighting (arcing) of the training set (see Breiman [1998b],

• p. 26 References

  Breiman, L. [1998a], Arcing Classifiers, (discussion paper) Annals of Statistics,

Judge's note: Viewed p7 and p26. The text has 'Breiman [1998c]' for random noise in the outputs and 'Breiman [1998b]' for arcing. The list has only 1998a (Arcing Classifiers) and 1998b (Randomizing Outputs). The evident fix is 1998c to 1998b and 1998b to 1998a. The reference entry is also printed 'Breiman. L. [1998b]' with a period instead of a comma (cosmetic).

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 8 Section 3.1, fourth paragraph

  Wolpert and Macready [1996]

• p. 27 References

  Wolpert, D.H. and Macready, W.G. [1997]

• p. 10 Section 4, third paragraph

  in Schapire et al [1997]

• p. 27 References

  Schapire, R. ,Freund Y. Bartlett P. and Lee W. [1998]

Judge's note: Viewed p8 and p27; the year mismatch is as printed. The other items bundled here are handled elsewhere: Schapire 1997 vs 1998 under C024, and Breiman 1998b/1998c under C016.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 10 Section 4, third paragraph

  Schapire et al [1997]

• p. 27 References

  Schapire, R. ,Freund Y. Bartlett P. and Lee W. [1998]

Judge's note: Viewed p10 and p27; the text search shows 'Schapire et al [1998]' in the concluding section. Year mismatch as printed. Also checked: 'Breiman et al [1985]' and 'Friedman [1991]' are cited in the text and have no entry in the reference list on p26 to p27.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 16 Section 8, third paragraph

  the number of features giving the lowest test set error in the Section 5 and 6 experiments

• p. 16 Section 8

  in the Section 5 and 6 experiments

• p. 16 Section 8

  lowest test set error in the Section 5 and 6 experiments

Judge's note: Page 16 image shows the phrase. Section 4 (p.9) is Random Input Selection with Table 2; Section 5 holds Forest-RC with Table 3; Section 6 is the strength/correlation study on three data sets. The introduction's outline has the same one-section offset ("Sections 5 and 6 give empirical results"), which suggests stale numbering.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 28 Appendix I

  Proof of Theorem 9.1

• p. 21 Section 11

  Theorem 11.1 As the number of trees in the forest goes to infinity

• p. 21 Section 11

  Theorem 11.1

Judge's note: Page 28 image shows "Proof of Theorem 9.1" followed by an argument about averages of y-values over hyper-rectangles, which is the regression case. A search of the text layer finds theorems numbered 1.2, 2.3, 11.1 and 11.2 only. Theorem 11.1 in Section 11 is the regression convergence theorem. The label is a stale theorem number.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 23 Section 12, paragraph after Table 5

  They originated in Friedman [1991]

Only the authors can say what is right here.

Judge's note: Page 23 image shows the citation. The reference list on pp.26-27 (text layer) runs Amit, Bauer, Breiman, Dietterich, Freund, Grove, Ho, Kleinberg, Schapire, Tibshirani, Wolpert, with no Friedman entry. Kept separate from C033, which covers the Breiman citations.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 23 Section 12, paragraph after Table 5

  described in Breiman [1998]

Only the authors can say what is right here.

Judge's note: Page 23 image shows "Breiman [1998]" unsuffixed. The reference list in the text layer has only 1998a and 1998b entries for Breiman, so the citation does not say which is meant. Only the author can say which was intended.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 6 Equation (8), first line

  EΘ var(Θ) ≤EΘ(EX,Yrmg(Θ,X,Y))2 −s2

Only the authors can say what is right here.

Judge's note: Page 6 image shows the exponent 2 after the closing parenthesis of (E_{X,Y} rmg(Θ,X,Y)). Reworked: var(Θ)=E_{X,Y}rmg² − (E_{X,Y}rmg)², so E_Θ var(Θ) = E_ΘE_{X,Y}rmg² − E_Θ(E_{X,Y}rmg)² ≤ E_ΘE_{X,Y}rmg² − s² ≤ 1 − s². Counterexample to the printed line: degenerate Θ, two classes, accuracy 0.75, so rmg=±1 with mean 0.5=s; the left side is 1−0.25=0.75 and the printed right side is 0.25−0.25=0, i.e. 0.75 ≤ 0, which is false. The final bound ≤ 1−s² and Theorem 2.3 still hold, hence S3.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 17 Section 8, paragraph after Table 4

  with the two multiclass data sets, glass and ecoli, along with diabetes, least effected by the noise

• p. 16 Table 4, soybean row, Adaboost

  26.9

• p. 9 Table 1, soybean row, Classes

  19

• p. 17 paragraph after Table 4

  least effected by the noise

Only the authors can say what is right here.

Judge's note: Table 4 (pp.16-17) lists glass, breast cancer, diabetes, sonar, ionosphere, soybean, ecoli, votes and liver. Table 1 (p.9) gives 6 classes for glass, 8 for ecoli and 19 for soybean; the other six are two-class. So Table 4 has three multiclass sets, not two, and soybean is not among the least affected. No explanation found elsewhere in the text. The "effected" typo is judged under C037.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 10 Section 4, third paragraph

  For Adaboost, 50 trees were combined for the first three data sets and 100 for zip-code.

• p. 10 Section 4

  50 trees were combined for the first three data sets and 100 for zip-code

Judge's note: Viewed p10 and Table 1 on p9. The larger sets are letters, sat-images and zip-code. 'First three' plus zip-code is self-contradictory; 'first two' is evidently intended, matching the random-forest sentence just before it.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 27 Appendix I, proof of Theorem 1.2

  Let Nk be the number of times that ϕ(Θn)=k in the first N trials

• p. 27 Appendix I, displayed equation after 'By the Law of Large Numbers'

  Nk = 1

• p. 27 Appendix I, proof of Theorem 1.2

  Let Nk be the number of times that ϕ(Θn)=k in the first N

Only the authors can say what is right here.

Judge's note: Page 27 image confirms both displays. Reworking: with N_k the count, Σ_{n=1..N} I(φ(Θn)=k) = N_k, so (1/N)Σ I(φ(Θn)=k) = N_k/N, not N_k. The preceding identity (1/N)Σ_n I(h(Θn,x)=j) = (1/N)Σ_k N_k I(x∈S_k) needs the count. The left side of the Law-of-Large-Numbers display should therefore be N_k/N. The step is misprinted but the proof's conclusion stands once N_k/N is read.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send