Paper check About

Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program

Journal of the American Statistical Association, 2010 · paper · Economics

Applies synthetic control methods to comparative case studies, estimating the effect of California's 1988 tobacco control program, Proposition 99. It estimates annual per-capita cigarette sales were about 26 packs lower by 2000 than otherwise.

Checked: PDF from nber.org, fetched 5 Oct 2026. Page numbers are those of that PDF.

8 findings

Substance Section III.B: 'Proposition 99 was passed in November 1988, giving us 18 years of pre-intervention data', but the data start in 1970 and the paper's own pre-period is 1970-1988, which is 19 years.
  • p. 18 Section III.B, first paragraph
    giving us 18 years of pre-intervention data
  • p. 24 Section III.D
    during the period 1970-1988
  • p. 24 Section III.D, MSPE definition
    the period 1970-1988) is about 3
  • p. 24 MSPE definition
    the period 1970-1988

Judge's note: Page 18 says the panel covers 1970-2000 and gives '18 years of pre-intervention data'. Page 24 defines the pre-Proposition 99 MSPE over 'the period 1970-1988'. Table 1 (page 48) uses 1988 cigarette sales as a pre-period predictor, and the effect is estimated for 1989-2000. Nothing in the paper explains an 18-year count. 1970 through 1988 inclusive is 19 years.

Second check: another model agreed

Report a problem with this finding
Reference or label Table 1 labels the income predictor "Ln(GDP per capita)", but Section III.B and Appendix A define it as "per-capita state personal income (logged)".
  • p. 48 Table 1, first row
    Ln(GDP per capita)
  • p. 19 Section III.B
    per-capita state personal income (logged)
  • p. 27 Appendix A
    Per-capita state personal income (logged).

Judge's note: Table 1 (p.48) labels the first row "Ln(GDP per capita)". The text on p.19 lists the predictor as "per-capita state personal income (logged)". Appendix A (p.27) describes it the same way and names the Census Statistical Abstract as the source. Nothing in the paper mentions GDP as a predictor, so this is the same variable under the wrong label. The data are unaffected.

Second check: another model agreed

Report a problem with this finding
Reference or label Section III.C: 'Figure 2 plots the yearly estimates of the impacts of Proposition 99, that is, the yearly gaps…', and the next two references to 'Figure 2', should point to Figure 3 ('Per-Capita Cigarette Sales Gap Between California and Synthetic California').
  • p. 22 Section III.C, first paragraph
    Figure 2 plots the yearly estimates of the impacts of Proposition 99, that is, the yearly gaps
  • p. 40 Figure 3 caption
    Figure 3: Per-Capita Cigarette Sales Gap Between California and Synthetic California
  • p. 22 Section III.C, first paragraph
    The magnitude of the estimated impact of Proposition 99 in Figure 2 is substantial.
  • p. 22 Section III.C, first paragraph
    Figure 2 plots the yearly estimates of the impacts of Proposition

Judge's note: Page 22 cites Figure 2 for the yearly gaps three times. The Figure 2 caption is 'Trends in Per-Capita Cigarette Sales: California vs. synthetic California' (levels). The page 40 image shows Figure 3 is the gap plot. A grep finds no citation of Figure 3 anywhere in the text.

Second check: another model agreed

Report a problem with this finding
Reference or label "In Figure 6 we lower the cutoff even further ... the 19 remaining control states in Figure 6" refers to the twice-MSPE, 19-state plot, which is Figure 7.
  • p. 25 Section III.D, second paragraph
    In Figure 6 we lower the cutoff even further
  • p. 25 Section III.D, second paragraph
    the 19 remaining control states in Figure 6
  • p. 44 Figure 7 caption
    Placebo Gaps in 19 Control States (Discards States with Pre-Proposition 99 MSPE Two Times Higher than California's)
  • p. 44 Figure 7 caption
    Figure 7: Per-Capita Cigarette Sales Gaps in California and Placebo Gaps in 19 Control

Judge's note: Page 25: the first paragraph assigns Figure 6 to the five-times cutoff with 29 control states, which matches the Figure 6 caption. The next paragraph describes the twice-MSPE cutoff with 19 control states and again cites 'Figure 6' twice. The Figure 7 caption (p.44) reads '19 Control States (Discards States with Pre-Proposition 99 MSPE Two Times Higher than California's)'. A text search finds no citation of Figure 7 anywhere in the text. Both pointers in that paragraph should read Figure 7.

Second check: another model agreed

Report a problem with this finding
Reference or label Appendix B cites "Source: Barro and Lee (1994)" for the investment rate, but the reference list contains only "Barro, R.J. and Lee, J. (2000)".
  • p. 31 Appendix B data sources, Investment Rate
    Source: Barro and Lee (1994).
  • p. 35 References
    Barro, R.J. and Lee, J. (2000)
  • p. 35 References
    Barro, R.J. and Lee, J. (2000), “International Data on Educational Attainment

Judge's note: The p.31 image shows 'Barro and Lee (1994)' for the investment rate and 'Barro and Lee (2000)' for schooling. The p.35 reference list has a single Barro and Lee entry, dated 2000 (the educational attainment paper, CID WP 42). There is no 1994 entry, so the citation has no matching reference.

Second check: another model agreed

Report a problem with this finding
Reference or label Reference Farrelly, Evans and Sfekas (1999): "Tobacco Control, vol. 8, 272-227" gives an end page lower than the start page.
  • p. 36 References, Farrelly, Evans and Sfekas (1999)
    Tobacco Control, vol. 8, 272-227.
  • p. 36 References, Farrelly, Evans, Sfekas
    vol. 8, 272-227
  • p. 36
    vol. 8, 272-227.

Judge's note: The p.36 image confirms '272-227'. The range is impossible as printed. The likely intended range is 272-277, but that reading rests on outside knowledge.

Second check: another model agreed

Report a problem with this finding
Reference or label Rubin (2001) reference gives "Health Services and Outcomes Research Methodology, vol. 1, 169-188"; the published volume appears to be 2.
  • p. 37
    Rubin, D. B. (2001), “Using Propensity Scores to Help Design Observational Studies: Applic

Only the authors can say what is right here.

Judge's note: Rests on outside knowledge: the Crossref record (doi:10.1023/a:1020363010465) gives volume 2 for this article. I checked the page 37 crop and it prints "vol. 1, 169-188". Nothing in the files lets me check the volume, so the authors should confirm it.

Second check: another model agreed, code check

Report a problem with this finding
Wording Appendix C defines "R3t = Σ w*j(εjt − ε1t)", which has the opposite sign to the term +Σ wj(ε1t − εjt) in the decomposition it comes from (and to −Σ w*j(εjt − ε1t) in eq. 3).
  • p. 33 Appendix C, definition of R3t
    and R3t = Σ w*j(εjt − ε1t)
  • p. 32 Appendix C, decomposition
    + Σ wj(ε1t − εjt)
  • p. 33 Appendix C, definition of R3t
    R3t = Σ w∗j (εjt − ε1t)
  • p. 9 Equation (3)
    − Σ w∗j(εjt − ε1t)

Only the authors can say what is right here.

Judge's note: Images checked. On p.32 the decomposition of Y1t − Σ wj Yjt ends with '+ Σ wj(ε1t − εjt)'. In eq. (3) on p.9 the term is '− Σ w*j(εjt − ε1t)', which is the same thing. On p.33, R1t = λt(..)^{-1}λ̄K' Σ w*j ε̄jK and R2t = −λt(..)^{-1}λ̄K' ε̄1K match the expansion of −λt(..)(ε̄1K − Σ wj ε̄jK). R3t, however, is printed as Σ w*j(εjt − ε1t) = −Σ w*j(ε1t − εjt). With R3t as printed, R1t+R2t+R3t does not equal the left-hand side. Check: with θt = λt = 0, J = 1, w*2 = 1, ε1t = 1, ε2t = 0, the left-hand side is 1 but printed R3t = −1. The '≃' covers only the approximation in (2), not this exact term. The stated conclusion that R3t has mean zero holds for either sign, so the result stands.

Second check: another model agreed

Report a problem with this finding
22 minor (typos and the like; most checked by the judge only)
Reference or label Reference "Abadie, A. and Gardeazabal, J. (2003) ... American Economic Review, vol. 93, no. 1, 112-132" gives first page 112.
  • p. 35
    Abadie, A. and Gardeazabal, J. (2003), “The Economic Costs of Conflict: A Case Study of the

Only the authors can say what is right here.

Judge's note: Rests on outside knowledge: the published article (AER 93(1), doi:10.1257/000282803321455188) begins on page 113, not 112. The p.35 image confirms the printed '112-132'. The files cannot show the true first page.

Second check: another model agreed, code check

Report a problem with this finding
Reference or label Reference "Card, D. (1990), 'The Impact of the Mariel Boatlift on the Miami Labor Market', Industrial and Labor Relations Review, vol. 44, 245-257" gives volume 44.
  • p. 35
    Card, D. (1990), “The Impact of the Mariel Boatlift on the Miami Labor Market”, Industrial

Only the authors can say what is right here.

Judge's note: Rests on outside knowledge: the article appeared in ILR Review vol. 43, no. 2 (1990), not vol. 44. The p.35 image confirms the printed 'vol. 44'. The files cannot show the true volume.

Second check: another model agreed, code check

Report a problem with this finding
Wording Card (1990) is described as estimating 'the unemployment rates of native workers' (footnote 1, p.3) but as estimating 'native employment rates in Miami' (Section II.D, p.13).
  • p. 13 II.D
    to estimate native employment rates in Miami
  • p. 3 footnote 1
    estimate the un- employment rates of native workers

Judge's note: Both passages were checked on the page images. They describe the same study's outcome in different terms. The two measures are related labour-market outcomes, so this is a minor consistency point.

Second check: another model agreed

Report a problem with this finding
Wording Undefined index bound N is used for the unit set ('all i ∈ {1, . . . , N}', 'F_t = {Y_js, Z_js}_{1≤j≤N}', '{w*_j}_{2≤j≤N}'), but units are defined as i = 1,…,J+1, and N is also the superscript for the no-intervention outcome Y^N.
  • p. 7 Section II.B
    all i ∈ {1, . . . , N}
  • p. 10 Section II.B, below eq. (4)
    {w∗j}2≤j≤N
  • p. 34 Appendix C
    {w∗j}2≤j≤N is a deterministic
  • p. 34 Appendix C
    {w∗ j}2≤j≤N

Judge's note: Page 7 defines units i = 1,…,J+1 and in the same sentence uses 'all i ∈ {1, . . . , N}'. Page 10 has F_t indexed 1≤j≤N and {w*_j}_{2≤j≤N}, followed directly by eq. (5), which sums j=2 to J+1. Page 34 (Appendix C) repeats {w*_j}_{2≤j≤N}. A grep finds no definition of N as a count. N evidently stands for J+1, but it is never introduced and clashes with the Y^N superscript.

Second check: another model agreed

Report a problem with this finding
Wording The text says the predictors, including per-capita beer consumption, 'are averaged over the 1980-1988 period', while the Table 1 note says '(beer consumption is averaged 1984-1988)'.
  • p. 19 Section III.B
    These variables are averaged over the 1980-1988 period
  • p. 48 Table 1 note
    (beer consumption is averaged 1984-1988)
  • p. 19 III.B
    averaged over the 1980-1988 period
  • p. 48 Table 1 note
    beer consumption is averaged 1984-1988

Only the authors can say what is right here.

Judge's note: Page 19 lists beer consumption among the predictors and gives a single averaging window of 1980-1988. The Table 1 note on page 48 gives an exception of 1984-1988 for beer. Appendix A (the beer data source) says nothing about availability before 1984. The two descriptions disagree, and only the authors can say which window was used and why.

Second check: another model agreed

Report a problem with this finding
Wording The text claims 'the high balance on all smoking predictors (Table 1)' and that synthetic California 'accurately reproduces' the predictor values, but in Table 1 Ln(GDP per capita) is 10.08 for real California versus 9.86 for synthetic California, the same as the 38-state average (9.86).
  • p. 21 Section III.C
    Combined with the high balance on all smoking predictors (Table 1)
  • p. 21 Section III.C
    the synthetic California accurately reproduces the values that smoking prevalence and smoking prevalence predictor variables had in California
  • p. 48 Table 1, Ln(GDP per capita) row
    10.08

Only the authors can say what is right here.

Judge's note: Table 1 (page 48) shows close matches on the other predictors. The income row shows no improvement over the donor average. The 'all predictors' wording on page 21 therefore overstates the balance. The V-weight on income may be small, but the text does not say so.

Second check: another model agreed

Report a problem with this finding
Typo Footnote 2: 'cases studies are used to test hypotheses' should read 'case studies'.
  • p. 7 Footnote 2
    cases studies are used to test hypotheses

Judge's note: Seen in footnote 2 on the page 7 image.

Second check: none (judge only)

Report a problem with this finding
Typo Section II.B: 'Let T0 be number of pre-intervention periods' is missing 'the'.
  • p. 7 II.B
    Let T0 be number of

Judge's note: Seen on the page 7 image, Section II.B.

Second check: none (judge only)

Report a problem with this finding
Typo Footnote 7: 'similar to the exposed units in term of the values of X1' should read 'in terms of'.
  • p. 11 Footnote 7
    similar to the exposed units in term of the values of X1

Judge's note: Seen in the footnote 7 crop from page 11.

Second check: none (judge only)

Report a problem with this finding
Typo Footnote 8: 'the variables in in X0 and X1' repeats the word 'in'.
  • p. 12 Footnote 8
    the variables in in X0 and X1
  • p. 12 footnote 8
    variables in in X0 and X1
  • p. 12
    le variances of the variables in in X0 and X1 (normaliz

Judge's note: Seen in the footnote 8 crop from page 12; the duplication is in the typeset text, not an extraction artefact.

Second check: code check

Report a problem with this finding
Typo Section II.D: 'our ignorance about the the ability of the control group' repeats the word 'the'.
  • p. 13 Section II.D
    our ignorance about the the ability of the control group
  • p. 13 II.D
    ignorance about the the ability
  • p. 13
    ives from our ignorance about the the ability of the cont

Judge's note: Seen on the page 13 image.

Second check: code check

Report a problem with this finding
Typo Footnote 12: 'Health Education(20 percent)' lacks a space before the parenthesis; p.17: 'Evans, Farrelly, and Montgomery, (1999)' has a stray comma before the year.
  • p. 15 footnote 12
    Health Educa- tion(20 percent)
  • p. 17 III.A
    Evans, Farrelly, and Montgomery, (1999)

Judge's note: Both seen on the page images (crops C009-a and C009-b).

Second check: none (judge only)

Report a problem with this finding
Typo Section III.A: 'the anti-tobacco media campaign was first temporally eliminated and then reestablished' should read 'temporarily'.
  • p. 18 Section III.A, Fichtenberg and Glantz paragraph
    first temporally eliminated and then reestablished

Judge's note: Seen on the page 18 image.

Second check: none (judge only)

Report a problem with this finding
Typo Section III.C: 'smoking trends in California prior to the passage of Proposition 99 is best reproduced' has a plural subject with a singular verb ('are').
  • p. 21 Section III.C, Table 2 paragraph
    smoking trends in California prior to the passage of Proposition 99 is best reproduced

Judge's note: Seen on the page 21 image.

Second check: none (judge only)

Report a problem with this finding
Typo "Trade Openness: Export plus Imports as percentage of GDP" mixes singular 'Export' with plural 'Imports'.
  • p. 31 Appendix B data sources
    Export plus Imports as percentage of GDP

Judge's note: The text layer shows the wording as quoted. This is a trivial grammatical inconsistency: 'Exports plus Imports' would be parallel. The meaning is unaffected.

Second check: none (judge only)

Report a problem with this finding
Typo Appendix C: "restrict ourselves to the case that F = 1, so there is one unobserved factor" uses a symbol F that is never defined.
  • p. 32 Appendix C
    restrict ourselves to the case that F = 1, so there is one unobserved factor
  • p. 32 Appendix C, first paragraph
    we will restrict ourselves to the case that F = 1, so there is one unobserved factor

Judge's note: A text search finds 'F' as a symbol only at this location; elsewhere it occurs only in names. The model in Section II.B and at the start of Appendix C has a single factor λtμi, and no symbol is introduced for the number of factors. The following clause makes the meaning clear, so this is a minor notation gap.

Second check: none (judge only)

Report a problem with this finding
Typo Footnote 19: "may be correlated across units and in time because the presence of the term λtμi" is missing 'of'.
  • p. 32 Footnote 19
    correlated across units and in time because the presence of the term λtµi

Judge's note: The p.32 image confirms the wording 'because the presence of the term'. It should read 'because of the presence of'.

Second check: none (judge only)

Report a problem with this finding
Typo "as the discrepancies in equation in equation (5) go to zero" repeats 'in equation'.
  • p. 34 Appendix C, last sentence
    the discrepancies in equation in equation (5) go to zero
  • p. 34 Appendix C, last paragraph
    discrepancies in equation in equation (5)
  • p. 34
    equation in equation (5) go to zero

Judge's note: The p.34 image, last sentence, shows the duplicated words.

Second check: none (judge only)

Report a problem with this finding
Typo "Assume that there exist two finite constant λmin > 0 and λmax < ∞" should read 'constants'.
  • p. 34 Appendix C
    there exist two finite constant λmin > 0

Judge's note: The p.34 image confirms 'two finite constant'.

Second check: none (judge only)

Report a problem with this finding
Typo Reference Meinhardt et al. (1995): "Deutsches Institut für Wirtschaftfroschung" is misspelled; it should be 'Wirtschaftsforschung'.
  • p. 36 References, Meinhardt et al. (1995)
    Wirtschaftfroschung
  • p. 36 References, Meinhardt et al. (1995)
    Wirtschaftfroschung, Sonderheft 154

Judge's note: The p.36 image confirms the spelling 'Wirtschaftfroschung'. This is a transposition of 'or' plus a missing 's'.

Second check: none (judge only)

Report a problem with this finding
Typo Longo et al. (2001) reference lists the third author as "Krusea, R."; the surname appears to be Kruse.
  • p. 36 References, Longo et al.
    Krusea, R.

Only the authors can say what is right here.

Judge's note: Rests on outside knowledge: the third author of Longo et al. (2001), Tobacco Control 10, 267-272, is R. Kruse. I checked the page 36 crop and it reads "KRUSEA, R." as quoted. The trailing 'a' looks like a typo.

Second check: none (judge only)

Report a problem with this finding
Typo Table 1 row label "Cigarette sales Per capita 1988" capitalises 'Per', unlike the 1980 and 1975 rows ("per capita").
  • p. 48 Table 1
    Cigarette sales Per capita 1988

Judge's note: I checked the Table 1 image on p.48. The 1988 row reads "Per capita" with a capital P, while the 1980 and 1975 rows read "per capita". The inconsistency is cosmetic only.

Second check: none (judge only)

Report a problem with this finding
Something wrong with this page, or a finding we missed?