Equality of Opportunity in Supervised Learning
2016 · paper · AI and machine learning
Proposes a criterion for discrimination against a protected attribute in supervised learning, and shows how to adjust any learned predictor to remove it. It also examines limits of such measures and illustrates the approach with FICO credit scores.
Checked: PDF from arxiv.org, fetched 4 Oct 2026. Page numbers are those of that PDF.
12 findings
Substance p.5: Feldman et al.'s formalization of "limiting disparate impact" is printed as "Pr{Ŷ=1|A=0} ⩽ 0.8·Pr{Ŷ=1|A=1}". A limiting condition under the 80% rule would require ⩾.
- p. 5 Section 3, paragraph 2
⩽0.8 · Pr
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: under the 80% rule, the protected group's selection rate must be at least 80% of the other group's (Feldman et al. define having disparate impact as a ratio ≤ 0.8). The page 5 image shows ⩽. The paper's own framing also shows the problem: as printed, the condition is satisfied by never accepting anyone with A=0, which does not limit disparate impact. The likely cause is that the definition of having disparate impact was copied in place of the constraint.
Second check: another model agreed
Report a problem with this finding
Substance Eq. (4.5) gives the objective as "min γ0ℓ(1,0) + (1 − γ1)ℓ(0,1)" over γ ∈ ∩aDa. Here γ0 and γ1 are conditional FPR and TPR, and the class weights Pr{Y=0} and Pr{Y=1} are missing, so this is not Eℓ(Ỹ,Y). The subscripts on γ also index groups in Eq. (4.1) but coordinates here.
- p. 9 Deriving an optimal equalized odds threshold predictor
γ = (γ0,γ1) ∈∩aDa
- p. 10 Eq. (4.5)
γ0ℓ(1,0) + (1 −γ1)ℓ(0,1)
- p. 6 Eq. (4.1)
γa(Ŷ)
Only the authors can say what is right here.
Judge's note: Page 6 defines ℓ(ŷ,y) as a per-example loss and the goal as minimizing Eℓ(Ỹ,Y). Eq. (4.1) uses γa for the rate vector of group a, while p.9–10 use γ=(γ0,γ1) for the FPR/TPR coordinates. With zero diagonal losses, Eℓ = Pr{Y=0}γ0ℓ(1,0) + Pr{Y=1}(1−γ1)ℓ(0,1). I checked an example with Pr(Y=1)=0.9 and vertices (0,0), (0.2,0.8), (1,1), using 0-1 loss. The printed objective gives 1, 0.4 and 1, so it picks (0.2,0.8). The true losses are 0.9, 0.2 and 0.1, so the optimum is (1,1). Nothing in the paper says ℓ absorbs the class priors.
Second check: another model agreed
Report a problem with this finding
Substance Theorem 5.6 assumes only "that ℓ is a bounded loss function" but states the bound Eℓ(Ŷ,Y) ≤ Eℓ(Y*,Y) + 2√2·dK(R̂,R*). The proof uses "∥v∥2 ⩽ √2", which needs the loss to be scaled to [0,1]; for a general bounded loss the bound fails without a loss-scale factor.
- p. 11 Theorem 5.6
Assume that ℓ is a bounded loss function
- p. 12 Proof of Theorem 5.6, final paragraph
∥v∥₂ ⩽ √2
Only the authors can say what is right here.
Judge's note: I read p.11 (theorem) and p.12 (proof). Counterexample: A and Y independent and fair, X=Y, R*=Y, R̂ ≡ 1/2, ℓ = 10·I{ŷ≠y}. Then dK = 1 (for Y=1 and t in [1/2,1), |0−1| = 1). Y* = Y has loss 0. Any predictor derived from (R̂,A) is independent of Y and has loss 5, which exceeds 2√2 ≈ 2.83. Stating a unit bound on the loss, or including its scale in the bound, would fix this.
Second check: another model agreed
Report a problem with this finding
Substance Section 6.2 on page 15 prints "Ŷ(R∗,A) = I{R∗ > tA} where tA = t−A". Given "R∗ = R̃ + A" (page 14), the same predictor as I{R̃ > t} needs tA = t+A.
- p. 14 Section 6.2, last paragraph
namely an additive shift R∗ = R̃ + A
- p. 15 Section 6.2, first line
where tA = t−A
- p. 15 Section 6.2, first paragraph
bY (R∗,A) = I{R∗> tA} where tA = t−A
- p. 14 Section 6.2, last paragraph
namely an additive shift R∗= eR + A
Judge's note: Page 14 states R∗ = R̃ + A, and Scenario II on page 13 (R∗=X3, R̃=X3−A) agrees. Then R̃ > t ⇔ R∗ − A > t ⇔ R∗ > t + A. The page 15 image clearly prints "tA = t−A", which corresponds to R̃ > t−2A, a different predictor whose threshold depends on A. This is a sign error.
Second check: another model agreed
Report a problem with this finding
Substance Page 14 prints "we can construct such a joint distribution with X2 = R̃ and X3 = R̃". X3 should be R∗, as the next sentence's "X3 = X1 + X2" requires.
- p. 14 Section 6.1, paragraph after Proposition 6.1
with X2 = R̃ and X3 = R̃
- p. 14 Section 6.1, paragraph after Proposition 6.1
with X2 = eR and X3 = eR. The features are correlated with each other, with X3 = X1 + X2.
- p. 14 Section 6.1
with X2 = eR and X3 = eR
- p. 14 Section 6.1, paragraph after Proposition 6.1
X2 = eR and X3 = eR
Judge's note: The page 14 image shows a tilde (R̃) in both assignments, so this is not an extraction artefact. Scenario II defines R∗ = X3, and Scenario I has X1 = A and R̃ = X2. So X3 = X1 + X2 = A + R̃ = R∗. Setting X3 = R̃ = X2 would force X1 = 0, contradicting A ∈ {−1,1}.
Second check: another model agreed
Report a problem with this finding
Substance Page 14 states R* = R̃ + A, but p.15 then derives the same predictor as Ŷ(R*,A) = I{R* > t_A} "where t_A = t−A". Since R̃ > t ⇔ R* > t + A, the threshold should be t_A = t + A.
- p. 14 Section 6.2, additive relation between scores
R* = R̃ + A
- p. 15 Section 6.2, first sentence
t_A = t − A
Judge's note: Page 14 gives the additive shift R* = R̃ + A, and the threshold rule is on p.15. Then R̃ > t ⇔ R* − A > t ⇔ R* > t + A. The printed t − A amounts to R̃ > t − 2A, which with A ∈ {−1,1} gives different TPRs across groups and so is not the same equalized-odds predictor. This is a sign error.
Second check: another model agreed
Report a problem with this finding
Substance Page 15 prints "Instead of C0(t) = C1(t), for R∗ we have C0(t) = C1(t − 1)". This does not follow from R∗ = R̃ + A and the definition Ca(t) = (Pr{R > t | A=a, Y=0}, Pr{R > t | A=a, Y=1}).
- p. 15 Section 6.2, before Definition 6.3
for R∗ we have C0(t) = C1(t −1)
Only the authors can say what is right here.
Judge's note: Ca(t) is defined on page 8 with R > t. Let C̃ be the common ROC curve of R̃. With R∗ = R̃ + A, Ca(t) = C̃(t − a). Coding A in {0,1}: C0(t) = C̃(t) but C1(t−1) = C̃(t−2), so the correct relation is C0(t) = C1(t+1). Under the {−1,1} coding that Section 6 uses (page 13), the shift is 2. The printed identity fails under either coding. The right fix depends on which coding the C0/C1 labels refer to, so this is an author query.
Second check: another model agreed
Report a problem with this finding
Wording The proof of Proposition 4.4 replaces Pr{Ỹ=y′,Y=y | Ỹ=Ŷ} by the unconditional Pr{Ŷ=y′,Y=y}, giving "= Pr{Ŷ = y′, Y = y} Pr{Ỹ = Ŷ} + Pr{Ŷ = 1 − y′, Y = y} Pr{Ỹ ≠ Ŷ}". This is false for admissible derived predictors.
- p. 7 Proof of Proposition 4.4, last equality in the probability expansion
Pr{Ỹ = Ŷ}
- p. 7 Proof of Proposition 4.4, displayed derivation of Pr{Ỹ=y′, Y=y}
= Pr{Ŷ = y′, Y = y} Pr{Ỹ = Ŷ} + Pr{Ŷ = 1 − y′, Y = y} Pr{Ỹ ≠ Ŷ}
Only the authors can say what is right here.
Judge's note: I read the display on p.7. The last equality would need the event {Ỹ=Ŷ} to be independent of (Ŷ,Y). For a derived predictor this generally fails, because the probability of keeping Ŷ depends on (ŷ,a). Counterexample I checked: Ŷ=Y, Pr{Y=1}=1/2, Ỹ ≡ 1 (all four parameters equal 1, which is a valid derived predictor). Then Pr{Ỹ=1,Y=0} = 1/2, but the printed right side is 0·(1/2) + (1/2)·(1/2) = 1/4. Proposition 4.4 still holds, because Pr{Ỹ=y′,Y=y} = Σ Pr{Ỹ=y′|Ŷ=ŷ,A=a}·Pr{Ŷ=ŷ,A=a,Y=y} is linear in the four parameters. So the step is faulty but the stated result stands.
Second check: another model agreed
Report a problem with this finding
Wording The proof of Theorem 5.6 on page 12 prints "∥p∗−q∥₂² ⩽ ∥p∗−q0∥₂² + ∥p∗−q0∥₂² ⩽ 4·dK(R̂,R∗)²", repeating the q0 term where the second term should involve q1.
- p. 12 Proof of Theorem 5.6
∥p∗−q∥2 2 ⩽∥p∗−q0∥2 2 + ∥p∗−q0∥2 2
- p. 12 Proof of Theorem 5.6, displayed inequality
∥p∗−q∥2 2 ⩽∥p∗−q0∥2 2 + ∥p∗−q0∥2 2 ⩽4 · dK(bR,R∗)2
- p. 12 Proof of Theorem 5.6
∥p∗−q∥2 ⩽ ∥p∗−q0∥2 + ∥p∗−q0∥2 ⩽ 4 · dK
- p. 12 proof
∥p∗−q0∥2 + ∥p∗−q0∥2
Only the authors can say what is right here.
Judge's note: The page 12 image shows q0 in both terms. As printed, the first inequality says ∥p∗−q∥² ⩽ 2∥p∗−q0∥². That fails in general: if q0 = p∗ it would force q = p∗, but q is any point of L0∩L1 and can be far from p∗ when q1 is far. The intended term is ∥p∗−q1∥². Since each term is at most 2dK², the bound 4dK² and the theorem are unchanged. The step is faulty as printed but the stated result stands, so S3.
Second check: another model agreed
Report a problem with this finding
Wording In the proof of Theorem 5.6, "Now, take q ∈ L₀ ∩ L₁" assumes that the segment L1 from (0,0) to q1 meets the segment L0 from q0 to (1,1). Under the stated without-loss-of-generality assumptions these segments can be disjoint.
- p. 12 Proof of Theorem 5.6, geometric construction
Now, take q ∈ L₀ ∩ L₁.
Only the authors can say what is right here.
Judge's note: I read p.12. Check: q0=(0.5,0.8), q1=(0.51,0.7), p*=(0.5,0.75). Both points are above the diagonal, and q1 has the larger first coordinate. L1 = (0.51u, 0.7u) and L0 = (0.5+0.5s, 0.8+0.2s). Matching x-coordinates needs s ≤ 0.02, where L0 has y ≥ 0.8 while L1 has y ≤ 0.7, so the segments do not meet. In this case q1 lies in conv{(0,0),q0,(1,1)} ⊆ D̂0, so a case split repairs the argument and the bound survives. The step is faulty but the stated result stands. (Separately, the displayed inequality on the same page repeats ∥p*−q0∥ where q1 is presumably meant.)
Second check: another model agreed
Report a problem with this finding
Wording The proof of Proposition 6.1 writes Pr{Y=y|X2=x2} ∝ "exp(2yx2)", ∝ "exp(2y(x1 + x2))" and "Pr{Y = y|X3 = x3} ∝ exp(2yx3)". Under the stated model (y ∈ {−1,1}, X2 = Y + N(0,1), Pr{Y=y|A=a} = 1/(1+exp(−2ay))), these should be proportional to exp(yx2), exp(y(x1+x2)) and exp(yx3).
- p. 16 Proof of Proposition 6.1
∝exp(2yx2)
- p. 16 Proof of Proposition 6.1, display
∝ exp(2y(x1 + x2)).
- p. 16 Proof of Proposition 6.1, scenario II
Pr{Y = y|X3 = x3} ∝ exp(2yx3) by construction
- p. 16 Proof of Proposition 6.1, posterior from X2
exp(2yx₂)
Only the authors can say what is right here.
Judge's note: Model definitions are on pp.13–14 and the proof on p.16. 1/(1+exp(−2ay)) = e^{ay}/(e^{a}+e^{−a}), which is ∝ exp(ay) in y. The N(y,1) likelihood is ∝ exp(xy) in y, and Y is marginally uniform. The posterior odds are therefore e^{2x}, which corresponds to ∝ exp(yx); exp(2yx) would give odds e^{4x}. At x2 = 1 the true posterior is about 0.8808, while the printed form gives about 0.982. The factor is doubled consistently in both scenarios, so monotonicity, the optimal scores and the equivalence of the scenarios still hold.
Second check: another model agreed
Report a problem with this finding
Wording The proof of Proposition 6.1 says that if R = f(X2,X1) obeys equalized odds "it cannot depend on X1 … any dependency of f on X₁ would create a statistical dependency on A = X₁". A function can depend on X1 without creating any statistical dependence on A.
- p. 16 Proof of Proposition 6.1, first paragraph
any dependency of f on X₁ would create a statistical dependency on A = X₁
Only the authors can say what is right here.
Judge's note: I read p.16 and the model on p.13. Counterexample checked: R = X1·sin(2πX2) with X2 = Y + Z, Z ~ N(0,1), X1 = A ∈ {−1,1}. Given Y=y and A=a, R = a·sin(2π(y+Z)) = a·sin(2πZ), because y is an integer. Since Z and −Z have the same distribution, the law of R is the same for a = ±1, so R ⊥ A | Y even though f uses X1. The optimality claim for R̃ can still be argued differently (an equalized-odds rule can be replaced by one that ignores A without loss), so the step is faulty but the result stands.
Second check: another model agreed
Report a problem with this finding
30 minor (typos and the like; most checked by the judge only)
Wording Lemma 4.3 says "A predictor Ỹ is derived if and only if for all a ∈ {0,1}, we have γa(Ỹ) ∈ Pa(Ŷ)". Read literally, the 'if' direction is false; the proof only shows that the rate pairs achievable by derived predictors are exactly Pa.
- p. 5 Definition 4.1
Ỹ is independent of X conditional on (R,A).
- p. 6 Lemma 4.3
A predictor Ỹ is derived if and only if
- p. 6 Lemma 4.3 and its proof
A predictor eY is derived if and only if for all a ∈{0,1}, we have γa(eY ) ∈Pa(bY ).
Only the authors can say what is right here.
Judge's note: Looked at p.5 (Definition 4.1: a derived predictor is a possibly randomized function of (R,A) alone, independent of X given (R,A)) and p.6 (Lemma 4.3 and its proof). Reworking: take Ỹ = X_c, a coordinate of X that is a fair coin independent of (Y,A,Ŷ). Then γa(Ỹ) = (1/2,1/2), which lies on the diagonal between (0,0) and (1,1), so it is in Pa(Ŷ). Yet Ỹ is not independent of X given (Ŷ,A), so it is not derived. The proof establishes only (i) derived ⇒ γa ∈ Pa and (ii) every point of Pa is attained by some derived predictor. The way the lemma is used in (4.3) and Proposition 4.4 (the achievable-rate characterization) is unaffected.
Second check: another model agreed
Report a problem with this finding
Wording The proof of Proposition 4.4 writes E[ℓ(Ỹ,Y)] = Σ ℓ(y,y′)Pr{Ỹ = y′, Y = y}, which puts the true label first. Page 6 defines ℓ(ŷ,y) with the prediction first, so the term should be ℓ(y′,y).
- p. 6 Loss minimization paragraph
ℓ(ŷ,y) ∈ R which indicates the loss (or cost, or undesirability) of predicting ŷ when the correct label is y
- p. 7 Proof of Proposition 4.4
ℓ(y,y′)Pr{Ỹ = y′,Y = y}
- p. 7 Proof of Proposition 4.4, first displayed expectation
ℓ(y,y′)
Only the authors can say what is right here.
Judge's note: Page 6 defines ℓ(ŷ,y) as the loss of predicting ŷ when the correct label is y. Eq. (4.5) on p.10 also uses ℓ(1,0) for false positives, i.e. prediction first. The display on p.7 has the arguments reversed. For an asymmetric loss the printed sum is a different quantity. Example: Y fair, Ỹ ≡ 1, ℓ(1,0)=2, ℓ(0,1)=1. The true expectation is 1; the printed sum gives 1/2. The linearity conclusion of Proposition 4.4 is unaffected.
Second check: another model agreed
Report a problem with this finding
Wording p.8: "Ca(t) def= (Pr{R̂ > t | A=a,Y=0}, Pr{R̂ > t | A=a,Y=1})" uses R̂, although Section 4.2 works with the score R (the next sentence refers to "the conditional distributions R|A,Y").
- p. 8 Section 4.2, definition of C_a(t)
Pr{R̂ > t | A = a,Y = 0}
- p. 8 Section 4.2, definition of C_a(t)
Ca(t) def = (Pr{R̂ > t | A = a,Y = 0},Pr{R̂ > t | A = a,Y = 1})
Judge's note: The page 8 image shows a hat over R in the definition. Everywhere else, Section 4.2 uses R. R̂ is introduced only in Section 5, as an approximate regressor. This is a symbol inconsistency.
Second check: another model agreed
Report a problem with this finding
Wording Eq. (4.4) defines D_a = convhull{C_a(t): t ∈ [0,1]} using strict thresholds R > t with R ∈ [0,1]. If R has an atom at 0, the endpoint (1,1) is not in the image, even though the text treats D_a as running from (0,0) to (1,1).
- p. 8 Definition of C_a and Equation (4.4)
t ∈ [0,1]
- p. 9 Characterization of feasible threshold predictors
the intersection of the areas under the A-conditional ROC curves
Only the authors can say what is right here.
Judge's note: On p.8, C_a(t) uses Pr{R̂>t | …} with R ∈ [0,1] and t ∈ [0,1]. If Pr{R=0 | A=a} > 0, then for every t in [0,1] the TPR and FPR are below 1, so (1,1) is not in D_a. The constant predictor Ỹ=1 is still derived and has rates (1,1), and p.9 and the proof of Theorem 5.6 use the segment to (1,1). This is a boundary case; allowing t < 0 or adding (1,1) explicitly fixes it.
Second check: another model agreed
Report a problem with this finding
Wording Page 9 says that, since the space is two-dimensional, "such a predictor Ỹ can always be taken to be a mixture of two threshold predictors". This fails for interior points of D_a when the score is discrete; a general point may need three thresholds.
- p. 9 Deriving an optimal equalized odds threshold predictor
such a predictor Ỹ can always be taken to be a mixture of two threshold predictors
Only the authors can say what is right here.
Judge's note: I checked the counterexample. R takes values 3/4, 1/2, 1/4 with Y=0 probabilities (0, 1/2, 1/2) and Y=1 probabilities (1/2, 1/4, 1/4). The threshold ROC points are (0,0), (0,1/2), (1/2,3/4), (1,1). The point (1/4,1/2) is inside the hull: at x=1/4 the hull runs from y=1/4 to y=5/8. Segments through pairs of image points give y = 3/8, 1/4 or 5/8 at x=1/4, or x=0, so none passes through (1/4,1/2). Two thresholds are enough for boundary points, which is where the optimum lies, and for continuous (connected) ROC curves. The paper allows a general real-valued score, and the FICO scores are discrete.
Second check: another model agreed
Report a problem with this finding
Wording For equal opportunity, p.10 says "Assuming continuity of the conditional ROC curves … In this case, no randomization is necessary." Continuity alone does not ensure that deterministic thresholds are optimal; something like concavity of the ROC curves is needed.
- p. 10 Deriving an optimal equal opportunity threshold predictor
In this case, no randomization is necessary.
Only the authors can say what is right here.
Judge's note: I reworked the counterexample: A and Y are fair and independent, and loss is 0-1. Group 0 has piecewise-linear ROC vertices (0,0), (0.4,0.5), (0.5,0.9), (1,1); group 1 has (0,0), (0.1,0.5), (1,1). Both curves are continuous. The deterministic loss at common TPR y is [x0+x1+2−2y]/4. Its slopes are −1/4, 1/80 and 6/5 on the three pieces, so the minimum is 3/8 at y=0.5. Randomizing group 0 to (5/18, 1/2) gives a loss of 31/90, which is lower. The Figure 2 caption's convexity argument (the tangent-line gap) likewise assumes concave ROC curves.
Second check: another model agreed
Report a problem with this finding
Wording Proposition 5.2 claims that "for any oblivious property C, there exists a predictor Y*(R,A)" that is optimal among predictors satisfying C. The proof only shows that each predictor's joint distribution with (A,Y) can be reproduced from (R,A); it does not show that an optimum is attained.
- p. 10 Proposition 5.2 and its proof
any oblivious property C
Only the authors can say what is right here.
Judge's note: I read p.10. Counterexample checked: X constant, A independent of Y, Y ~ Bernoulli(1/4), 0-1 loss, and C: Pr(Ŷ=1) > 0. C is oblivious because it depends only on the distribution of Ŷ. Every predictor is independent of Y, so its loss is 1/4 + r/2 with r > 0. The infimum 1/4 is not attained, so no optimal predictor exists. The distribution-preserving reduction in the proof is correct, and Corollary 5.3 for equalized odds (a closed constraint) is not affected by this example.
Second check: another model agreed
Report a problem with this finding
Wording The proof of Theorem 5.6 on page 12 says Y∗ "is an optimal equalized odds predictor as required by the lemma". The result being proved is Theorem 5.6, not a lemma.
- p. 12 Proof of Theorem 5.6, para 2
as required by the lemma
- p. 12 proof of Theorem 5.6
optimal equalized odds predictor as required by the lemma
Judge's note: The page 12 image confirms the wording. Theorem 5.6 (page 11) asks for "an optimal equalized odds predictor Y∗". Lemma 5.5 makes no such requirement, so "the lemma" can only mean the theorem. This is a mislabel in wording only.
Second check: another model agreed
Report a problem with this finding
Wording Fig. 9 caption: "within each race, the equal opportunity threshold and average equal odds threshold lie between the max profit threshold and equal demography thresholds"; 'average equal odds threshold' is undefined, and under the midpoint reading it does not hold for White.
- p. 19 Figure 9 caption
within each race, the equal opportunity threshold and average equal odds threshold lie between the max profit threshold and equal demography thresholds
- p. 18 Section 7, paragraph after equalized-odds bullet
Our proposed fairness definitions give thresholds between those of max-profit/race-blind thresholds and of demographic parity.
Only the authors can say what is right here.
Judge's note: Read approximately from the Fig. 9 image (p.19). White raw: max profit ≈620, demography ≈668, equal-odds range ≈520–672, so the midpoint ≈596 is below max profit. Within-group: range ≈18–49 (midpoint ≈33.5) against max profit ≈35. The other groups are consistent with the caption. The caption says the equal-odds range is 'not uniquely defined' and does not say how 'average' is computed. A probability-weighted average might satisfy the claim, so this is an ambiguity for the authors to resolve rather than an established error.
Second check: another model agreed
Report a problem with this finding
Typo p.2: "requires subjective interpretation or casual assumptions about specific features" should read "causal assumptions".
- p. 2 Introduction, para on oblivious notion
subjective interpretation or casual assumptions
Judge's note: The page 2 image shows "casual". The context, and the later discussion of causal structure in Section 6, show that "causal" is meant.
Second check: none (judge only)
Report a problem with this finding
Typo "then" is printed for "than" in several places: p.2 "target rather then the protected attribute"; p.8 "rather then with a binary predictor"; p.15 "rather then protecting discrimination"; p.21 "more biased then the target label" and "correlated with A then Y".
- p. 2 Contributions bullet 1
target rather then the protected attribute
- p. 8 Section 4.2 para 1
rather then with a binary
- p. 15 Para before equal precision
rather then protecting discrimination
- p. 21 Predictive affirmative action
more biased then the target label
Judge's note: Each instance appears as quoted in the page 2 and page 8 images and in the crops for pages 15 and 21. Each is a comparative, so "than" is required.
Second check: none (judge only)
Report a problem with this finding
Typo p.5: "In practice, however, in can be difficult to come up with a suitable metric" should read "it can be difficult".
- p. 5 Section 3, para 3
in can be difficult to come up with
Judge's note: The page 5 image confirms that "in" is printed.
Second check: none (judge only)
Report a problem with this finding
Typo p.6, Section 4.1: "A is a binary protected attribute The proof generalizes" is missing a full stop.
- p. 6 Section 4.1, first sentence
A is a binary protected attribute The proof generalizes
Judge's note: The page 6 image confirms there is no full stop.
Second check: none (judge only)
Report a problem with this finding
Typo p.8: "The corresponding optimization problem for equation opportunity" should read "equal opportunity".
- p. 8 First line
optimization problem for equation opportunity
- p. 8 top of page
The corresponding optimization problem for equation opportunity
Judge's note: The page 8 image confirms that "equation" is printed.
Second check: none (judge only)
Report a problem with this finding
Typo p.8: "We will therefor consider" should read "therefore".
- p. 8 Section 4.2
We will therefor consider
Judge's note: The page 8 image confirms that "therefor" is printed.
Second check: none (judge only)
Report a problem with this finding
Typo p.8: "ROC (Receiver Operator Characteristic)"; the usual expansion is "Receiver Operating Characteristic".
- p. 8 Section 4.2
ROC (Receiver Operator Characteristic)
Judge's note: The page 8 image confirms the wording. "Operator" is a common variant, so this is only a minor consistency point.
Second check: none (judge only)
Report a problem with this finding
Typo p.8: "the horizontal axes is the false positive rate ... and the vertical axes is the true positive rate" should use the singular "axis".
- p. 8 4.2
the horizontal axes is the false
Judge's note: The page 8 image confirms both instances.
Second check: none (judge only)
Report a problem with this finding
Typo Page 9 prints "with a (possible randomized) predictor". It should read "possibly randomized".
- p. 9 Deriving an optimal equalized odds threshold predictor
with a (possible randomized) predictor
Judge's note: I checked the crop of page 9 and the text is printed as quoted. The adjective "possible" should be the adverb "possibly". This is a cosmetic grammar slip.
Second check: none (judge only)
Report a problem with this finding
Typo The opening of Section 5 on page 10 reads "we develop the theory a theory for non-discriminating Bayes optimal classification", with a redundant duplicated phrase.
- p. 10 Section 5 intro
we develop the theory a theory for non-discriminating
- p. 10 Section 5, first sentence
we develop the theory a theory for non-discriminating Bayes optimal
Judge's note: The full image of page 10 shows the duplicated phrase "the theory a theory". It is cosmetic.
Second check: none (judge only)
Report a problem with this finding
Typo Section 5 on page 10 has the wrong article in "can be obtained as an derived threshold predictor". It should be "a derived".
- p. 10 Section 5
as an derived threshold predictor
Judge's note: Page 10 prints "an derived". This is a cosmetic article error.
Second check: none (judge only)
Report a problem with this finding
Typo The proof of Lemma 5.5 on page 11 thresholds R "at t ∈ [0,1]" and then thresholds R′ "at the same threshold t′". The same threshold gets two names.
- p. 11 Proof of Lemma 5.5
thresholding R′ at the same threshold t′
Judge's note: On the page 11 image the proof introduces t and then writes "the same threshold t′". The intended meaning (same t) is clear, so this is a cosmetic notation slip.
Second check: none (judge only)
Report a problem with this finding
Typo Scenario II on page 13 has an extra "is" in "so as to make R̃ is independent of A given Y".
- p. 13 Scenario II
so as to make R̃ is independent of A given Y
- p. 13 Scenario II
so as to make eR is independent of A given Y
Judge's note: The page 13 image shows the extra "is". This is a cosmetic grammar slip.
Second check: none (judge only)
Report a problem with this finding
Typo Page 14 prints "Suppose we have access to all three feature". It should be "features".
- p. 14 Para after Proposition 6.1
to all three feature, i.e.
- p. 14 6.1
access to all three feature
Judge's note: The page 14 image shows the singular "feature". This is a cosmetic slip.
Second check: none (judge only)
Report a problem with this finding
Typo "the dependency structure implies its optimal even if we allow dependence on A" — "its" should be "it's"/"it is".
- p. 16 Proof of Proposition 6.1
implies its optimal even if we allow
Judge's note: Seen on the p.16 image, in the Scenario II paragraph of the proof.
Second check: none (judge only)
Report a problem with this finding
Typo Duplicated word: "i.e., for whom whom false positives (giving loans to people that default...)".
- p. 17 Section 7, para after Figure 7
for whom whom false positives
- p. 17
t rates below this, i.e., for whom whom false positives (gi
Judge's note: The p.17 page image shows "whom whom", so this is not an extraction artefact.
Second check: code check
Report a problem with this finding
Typo "The cost is thus born by the company" should read "borne by".
- p. 19 Section 7, para 3 after Fig 9
The cost is thus born by the company
Judge's note: Seen on p.19, in the paragraph after Fig. 9.
Second check: none (judge only)
Report a problem with this finding
Typo "more biased then the target label (roughly, R* is more correlated with A then Y is correlated with A)": 'then' should be 'than' in both places.
- p. 21 Conclusions, Predictive affirmative action
more biased then the target label
- p. 21 Conclusions, Predictive affirmative action
correlated with A then Y is correlated with A
Judge's note: Both instances are visible on the p.21 image, in the 'Predictive affirmative action' paragraph.
Second check: none (judge only)
Report a problem with this finding
Typo Reference [CKP09]: "In In Proc. IEEE International Conference on Data Mining Workshops" has a duplicated 'In'.
- p. 22 References, [CKP09]
In In Proc. IEEE International Conference
- p. 22 References, [CKP09]
In In Proc. IEEE International Conference on Data Mining
Judge's note: Visible on the p.22 reference list image.
Second check: none (judge only)
Report a problem with this finding
Typo Reference [PPM+14] lists "Jefrey Zients"; the name is presumably 'Jeffrey Zients'.
- p. 22 References, [PPM+14]
Jefrey Zients
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that the co-author's name is spelled 'Jeffrey Zients'. The p.22 image does read 'Jefrey Zients'. The correct spelling cannot be checked in the files, so this goes to the authors.
Second check: none (judge only)
Report a problem with this finding
Typo [BZVGRG15] "CoRR, abs:1507.05259, 2015" uses 'abs:' where the other CoRR entries use 'abs/' (e.g. abs/1609.05807, abs/1511.00830, abs/1505.05723).
- p. 22 References, [BZVGRG15]
CoRR, abs:1507.05259, 2015
- p. 22 References, [BZVGRG15]
CoRR, abs:1507.05259
- p. 22 [BZVGRG15]
abs:1507.05259
Judge's note: Checked on the p.22 image. Only this entry uses a colon, so it is a cosmetic inconsistency within the reference list.
Second check: none (judge only)