A Unified Approach to Interpreting Model Predictions
2017 · paper · AI and machine learning
Presents SHAP, a framework assigning each feature an importance value for a prediction. It unifies six existing explanation methods, shows a unique solution with desirable properties, and proposes new estimation methods that the authors say agree better with human intuition.
Checked: PDF from arxiv.org, fetched 4 Oct 2026. Page numbers are those of that PDF.
3 findings
Substance Theorem 1, Eq. 8 weights fx(z′) − fx(z′ \ i) by "|z′|!(M −|z′| −1)!/M!", but a non-zero term needs z′_i = 1, so |z′| = |S|+1 and the Shapley weight from Eq. 4 is (|z′|−1)!(M−|z′|)!/M!.
- p. 4 Theorem 1, Equation 8
|z′|!(M −|z′| −1)!
- p. 3 Section 2.4, Equation 4
|S|!(|F| −|S| −1)!
Only the authors can say what is right here.
Judge's note: Checked the crops of Eq. 8 (p. 4) and Eq. 4 (p. 3). My own working: Property 3 defines z′ \ i as setting z′_i = 0, so the bracket fx(z′) − fx(z′ \ i) is zero unless z′ contains i. Then S = z′ \ i has |S| = |z′| − 1, and the Eq. 4 weight |S|!(M−|S|−1)!/M! becomes (|z′|−1)!(M−|z′|)!/M!. Counterexample: M = 1, x′ = (1). The only contributing z′ is (1), where the printed weight contains (1−1−1)! = (−1)!, which is undefined; the Shapley value should be f(1) − f(0). For M = 2, the printed weight for |z′| = 1 is 1/2, and for |z′| = 2 it is again undefined. With the corrected weights, both terms are 1/2. The theorem says |z′| is the number of non-zero entries in z′, so it cannot be read as the coalition size without i. The formula as printed is wrong; the intended Shapley result stands once the weight is corrected.
Second check: another model agreed
Report a problem with this finding
Substance Corollary 1 (Linear SHAP) gives "φ0(f, x) = b" together with φi = wi(xi − E[xi]). The sum φ0 + Σφi is then f(x) − Σ wj E[xj], not f(x), so local accuracy fails unless E[x] = 0. The paper's own base value is φ0 = fx(∅) = E[f(z)] = b + Σ wj E[xj].
- p. 6 Corollary 1 (Linear SHAP)
φi(f, x) = wj(xj −E[xj])
- p. 6 Corollary 1
φ0(f, x) = b and
- p. 5 Figure 1 caption
base value E[f(z)] that would be predicted if we did not know any features
Only the authors can say what is right here.
Judge's note: Checked the Corollary 1 crop (p. 6), the Figure 1 caption (base value E[f(z)], p. 5), and p. 6, which says the infinite kernel weight "enforces φ0 = fx(∅)". For a linear f, E[f(z)] = b + Σ wj E[xj]. My own check: M = 1, w = 1, b = 0, E[x] = 1, x = 3. The corollary gives φ0 = 0 and φ1 = 2, which sum to 2, while f(x) = 3, so Property 1 fails. Using φ0 = E[f] = 1 restores it. The corollary assumes nothing about centred features. The separate i/j index mismatch raised by some reviewers is judged under C006.
Second check: another model agreed
Report a problem with this finding
Wording In Theorem 2 the loss is printed as "L(f, g, πx′) = Σ [f(h_x^{−1}(z′)) − g(z′)]² πx′(z′)", but the paper defines x = hx(x′) as the map from simplified to original inputs, so the model term should be f(hx(z′)).
- p. 6 Theorem 2, loss function L
f(h−1 x (z′)) −g(z′)
- p. 2 Section 2 and Section 2.1
g(z′) ≈f(hx(z′))
- p. 2 Section 2, definition of h_x
mapping function x = hx(x′)
Judge's note: The crop of p. 6 shows a superscript −1 on h_x, so this is not an extraction artefact. Page 2 defines "a mapping function x = hx(x′)" and "g(z′) ≈ f(hx(z′))". Property 3 and Section 4 use fx(z′) = f(hx(z′)) consistently, and the article never defines an inverse convention. Applying h_x^{−1} to a simplified input z′ contradicts these definitions. The intended reading (f(hx(z′))) is clear, so this is a notation slip in the theorem statement.
Second check: another model agreed
Report a problem with this finding
9 minor (typos and the like; most checked by the judge only)
Reference or label Section 5: "other methods that fail to meet Properties 1-3 (Section 2)" — Properties 1-3 are stated in Section 3, not Section 2.
- p. 8 Section 5, opening paragraph
Properties 1-3 (Section 2)
- p. 4 Section 3
Property 1 (Local accuracy)
- p. 8 Section 5, opening paragraph
other methods that fail to meet Properties 1-3 (Section 2)
- p. 8 Section 5 intro
fail to meet Properties 1-3 (Section 2)
Only the authors can say what is right here.
Judge's note: Viewed pages 8, 4 and 2. Properties 1 (local accuracy), 2 (missingness) and 3 (consistency) are all in Section 3 on page 4; Section 2 holds Definition 1 and the survey of existing methods. As placed, the parenthesis follows 'Properties 1-3' and reads as a wrong section pointer. It could instead be meant to point at the 'other methods' described in Section 2, so the authors should say which was intended; hence a query, not a plain correction.
Second check: another model agreed
Report a problem with this finding
Wording Section 2.2 introduces the DeepLIFT contribution as "a value C∆xi∆y" but Equation 3 and the text after it write "C∆xi∆o"; y is never defined in the section (only o = f(x)), and Equation 3 sums to n where the paper otherwise uses M.
- p. 3 Section 2.2, first paragraph
a value C∆xi∆y
- p. 3 Section 2.2, Equation 3
C∆xi∆o = ∆o
Judge's note: Viewed page 3. The prose has subscript Δy; Equation 3 and the following sentence (φi = C_{Δxi Δo}) have Δo. Only o is defined. The upper limit of the sum in Equation 3 is n, while Definition 1 on page 2 uses M for the number of simplified inputs. Notational inconsistency with an evident fix (Δo); meaning is not in doubt.
Second check: another model agreed
Report a problem with this finding
Typo The class defined as "Additive feature attribution methods" (Definition 1) is called "a new class of additive feature importance measures" in the abstract and "the class of additive feature importance methods (which includes six previous methods)" in the conclusion.
- p. 9 Section 6, first paragraph
the class of additive feature importance methods (which includes six previous methods)
- p. 1 Abstract
a new class of additive feature importance measures
- p. 2 Definition 1
Additive feature attribution methods
Judge's note: Viewed pages 1, 2 and 9. All three wordings are printed as quoted and refer to the same class (the one that unifies six methods). The terms are evidently used as synonyms and the meaning does not change; this is a terminology consistency point only.
Second check: none (judge only)
Report a problem with this finding
Typo Section 2.3: "this menthod is equivalent to DeepLIFT" — 'menthod' should be 'method'.
- p. 3 Section 2.3
this menthod is equivalent to DeepLIFT
- p. 3 Section 2.3
this menthod is equivalent
- p. 8 Section 5.1
Kernal SHAP
Judge's note: Viewed page 3; the misspelling is printed in the Section 2.3 text. The 'Kernal SHAP' misspelling that some reviewers attached here is a separate defect, handled under C009.
Second check: none (judge only)
Report a problem with this finding
Typo Corollary 1 prints "φi(f, x) = wj(xj −E[xj])": the left side is indexed by i, the right by j, which is the bound summation index of f(x) = Σ_{j=1}^M wj xj + b. It should read wi(xi − E[xi]).
- p. 6 Corollary 1 (Linear SHAP)
φi(f, x) = wj(xj −E[xj])
Judge's note: The crop of p. 6 confirms the printed indices. j is introduced only as the summation index in the model definition, so the right-hand side should use i. The intended meaning is clear, so this is a typographical slip.
Second check: none (judge only)
Report a problem with this finding
Typo Eq. 13 prints "m_{xj f3} = φi(f3, x)/(xj − E[xj])": the numerator is indexed by i while the multiplier and denominator use j. By analogy with Eq. 14 it should be φj(f3, x).
- p. 7 Deep SHAP, Equation 13
mxjf3 = φi(f3, x) xj −E[xj]
- p. 7 Deep SHAP, Equation 13
mxjf3 = φi(f3, x) / (xj −E[xj])
- p. 7 Equation 13
m_{x_j f_3} = φ_i(f_3, x) / (x_j − E[x_j])
- p. 7 Equation 13
φi(f3, x)
Judge's note: The crop of p. 7 confirms the subscript i in the numerator of Eq. 13. The ∀j∈{1,2} quantifier is attached to Eq. 14. In Eqs. 14–16, i indexes the y inputs and j indexes the x inputs (Eq. 15 sums over j of m_{yi fj} m_{xj f3}). The multiplier for xj therefore needs φj in the numerator. This is an index typo; the method itself is unaffected.
Second check: none (judge only)
Report a problem with this finding
Typo Figure 2 caption run-on: "...when all possible z′ vectors are ordered by cardinality there are 2^15 vectors in this example." — two clauses joined with no punctuation or conjunction.
- p. 7 Figure 2 caption
ordered by cardinality there are 215 vectors in this example
- p. 7 Figure 2 caption
are ordered by cardinality there are 2
Judge's note: Viewed page 7. The caption prints the two clauses with no separator; no dash or parenthesis is present in the image. The '215' in the text layer is 2^15 on the page, which is an extraction artefact and not part of this finding.
Second check: none (judge only)
Report a problem with this finding
Typo Section 5.1: "Kernal SHAP uses this connection to compute feature importance" — 'Kernal' should be 'Kernel'.
- p. 8 Section 5.1
Kernal SHAP
Judge's note: Viewed page 8. 'Kernal SHAP' is printed in the first paragraph of Section 5.1; the method is 'Kernel SHAP' elsewhere, including the Section 5 opening paragraph and the Figure 3 caption on the same page.
Second check: none (judge only)
Report a problem with this finding
Typo Acknowledgements: "supported by a National Science Foundation (NSF) DBI-135589" — the article 'a' has no noun (such as 'grant') to govern.
- p. 10 Acknowledgements
supported by a National Science Foundation (NSF) DBI-135589
Judge's note: Viewed page 10. The text is printed as quoted, followed by a list mixing grant numbers and a fellowship. It can be read with 'grant' implied, so this is a cosmetic wording point at most.
Second check: none (judge only)