The Experimental Approach to Development Economics
Annual Review of Economics, 2009 · paper · Economics
Reviews the literature on randomized experiments in development economics, weighing their strengths and limitations. Argues their main virtue is allowing estimation of parameters otherwise hard to evaluate, and that criticisms raised against them are real but often not specific to experiments.
Checked: PDF from economics.mit.edu, fetched 5 Oct 2026. Page numbers are those of that PDF.
9 findings
Substance Cost per extra child-year of education from deworming is given as "$3.25 with deworming" and "ranges from $3.25 to more than $200", but as "between $3.50 per extra child-year for deworming and $6000" in between, all on PDF page 3.
- p. 3 first paragraph
child-year of education was $3.25 with deworming
- p. 3 second paragraph
between $3.50
- p. 3 second paragraph
induced ranges from $3.25 to more than $200
- p. 3 para 2 of Sec. 2
$3.50 per extra child-year for deworming
Only the authors can say what is right here.
Judge's note: Page 3 image read. All three figures are for the cheapest programme (deworming). The $3.50 is attributed to the J-PAL (2005) bulletin, but the range two sentences later, from the same comparison, starts at $3.25, and no explanation of a different cost base is given anywhere in the article. Only the authors can say which figure is meant.
Second check: another model agreed
Report a problem with this finding
Reference or label "in the programs evaluated in Angrist et al. (2008) and Angrist & Lavy (2009)" (PDF p. 6): there is no Angrist et al. 2008 entry in Literature Cited; the entries are 2002, 2006 and 2009 (Angrist, Lang & Oreopoulos), the last cited as "Angrist et al. 2009" on PDF p. 5.
- p. 6 second paragraph
in the programs evaluated in Angrist et al. (2008) and Angrist & Lavy (2009)
- p. 25 Literature Cited
Angrist J, Lang D, Oreopoulos P. 2009. Incentives and services for college achievement
- p. 5 second paragraph
Angrist et al. 2009
- p. 6 Sec. 2, Berry paragraph
Angrist et al. (2008) and Angrist & Lavy (2009)
Only the authors can say what is right here.
Judge's note: Page 6 image and the reference list on the page 25 image read. Angrist entries: Acemoglu & Angrist 2001, Angrist et al. 2002, 2006, Angrist, Lang, Oreopoulos 2009, Angrist & Lavy 2009. Probably the 2009 student-incentives paper with a wrong year, but the authors must confirm.
Second check: another model agreed
Report a problem with this finding
Reference or label "An example is Bertrand et al. 2009, who set up an experiment to understand the structure of corruption in the process of obtaining a driving license in Delhi": the reference list has this study as Bertrand, Djankov, Hanna, Mullainathan 2007; the 2009 entry is the advertising-content paper, in which "Mulainathan" is also misspelled.
- p. 7 last paragraph
An example is Bertrand et al. 2009
- p. 25 Literature Cited
Bertrand M, Djankov S, Hanna R, Mullainathan S. 2007. Corruption in driving licensing process in
- p. 25 Literature Cited
Bertrand M, Karlan D, Mulainathan S, Zinman J. 2009. What’s advertising content worth?
- p. 7 Sec. 2, institutions paragraph
An example is Bertrand et al. 2009,
Judge's note: Page 7 and page 25 images read. The list has Bertrand et al. 2005 (implicit discrimination), 2007 ("Corruption in driving licensing process in Delhi", Q. J. Econ. 122(4)) and 2009 ("What's advertising content worth?"). The citation on page 7 therefore points to the wrong entry; page 23 uses Bertrand et al. (2009) for the advertising study. "Mulainathan" in the 2009 entry is spelled Mullainathan in the two entries above it.
Second check: another model agreed
Report a problem with this finding
Reference or label PDF p19: "the distribution of test scores among the students who study in schools that received a balsakhi ... first-order stochastically dominates that of the treatment group" - the comparison group is mislabelled; the balsakhi schools are the treatment group (should be comparison or control group).
- p. 19 Section 3.5, second paragraph
first-order stochastically dominates that of the treatment group
- p. 20 Section 3.5, paragraph beginning 'Moreover'
the distribution of test scores in treatment first-order stochastically dominated that in control
- p. 20 Sec. 3.5, balsakhi passage
distribution of test scores in treatment first-order stochastically domi-
- p. 20 Section 3.5
in treatment first-order stochastically domi- nated that in control
Judge's note: Page image p19 prints "dominates that of the treatment group". Schools that received a balsakhi are the treated schools, so the sentence compares treatment with itself. Page 20 states the same result as "the distribution of test scores in treatment first-order stochastically dominated that in control". Nothing in the text gives "treatment group" another meaning here. The direction of the result is recoverable from context, so this is a mislabelled population.
Second check: another model agreed
Report a problem with this finding
Reference or label "Manski C. 2004. Statistical treatment rules for heterogeneous populations. Econometrica. 2 (4):1221–46" gives volume 2 for a 2004 article; the list's other 2004 Econometrica entries are volume 72.
- p. 27 Literature Cited
Econometrica. 2 (4):1221–46
- p. 27 Literature Cited
Econometrica. 72(1):159–217
Judge's note: Page 27 image shows "Econometrica. 2" at the end of the line and "(4):1221–46" on the next, so the single digit is in the print and is not an extraction loss. On the same page Miguel & Kremer 2004 is "Econometrica. 72(1):159–217", and on page 26 Duflo & Chattopadhyay 2004 is "Econometrica 72(5):1409–43". A digit has been dropped: 72(4).
Second check: another model agreed
Report a problem with this finding
Reference or label "Imbens G, Angrist J. 1994. Identification and estimation of local average treatment effects. Econometrica 61(2):467–76": volume 61 does not fit 1994 by the list's other Econometrica entries (66 for 1998, 72 for 2004), which give 62.
- p. 27 Literature Cited
trica 61(2):467–76
- p. 27 Literature Cited
Econometrica 66:1017–98
- p. 27 Literature Cited
data. Econometrica 66:1017–98
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that Econometrica publishes one volume per calendar year, so the volume can be inferred from the year. Page 27 image shows the entry as quoted, and Heckman et al. 1998 as "Econometrica 66:1017–98", Miguel & Kremer 2004 as volume 72. Counting back from either gives 62 for 1994. The text also cites the paper as 1994, so the volume is the likelier wrong element, but the true volume is not in the files.
Second check: another model agreed
Report a problem with this finding
Reference or label "Hirano K, Porter J. 2005. Asymptotics for statistical decision rules. Econometrica 71(5):1307–38": year 2005 and volume 71 do not fit together by the list's own entries (volume 72 is 2004).
- p. 27 Literature Cited
Hirano K, Porter J. 2005. Asymptotics for statistical decision rules. Econometrica 71(5):1307–38
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that Econometrica publishes one volume per calendar year. Page 27 image shows the entry as quoted. The list has two 2004 entries in volume 72, so volume 71 would be 2003 and a 2005 article would be volume 73. Year and volume cannot both be right; which element is wrong (possibly a working-paper year joined to guessed journal details) cannot be settled from the files.
Second check: another model agreed
Report a problem with this finding
Reference or label "Thornton R. 2007. The demand for and impact of HIV testing. Evidence from a field experiment. Am. Econ. Rev. 98(5):1829–63": volume 98 is 2008 in the list's other Am. Econ. Rev. entries, so year and volume do not fit; the text also cites "Thornton 2007".
- p. 28 Literature Cited
Econ. Rev. 98(5):1829–63
- p. 27 Literature Cited
nance. Am. Econ. Rev. 98(3):1040–68
- p. 28 Literature Cited
Thornton R. 2007. The demand for and impact of HIV testing. Evidence from a field experiment. Am.
Only the authors can say what is right here.
Judge's note: Page 28 image shows the entry as quoted. Elsewhere in the list Am. Econ. Rev. volume 98 is dated 2008 (Karlan & Zinman 2008, 98(3):1040–68 on page 27; Duflo, Kremer & Robinson 2008c, 98(2):482–88 on page 26) and volume 96 is 2006 (Todd & Wolpin 2006, 96(5); Angrist et al. 2006, 96(3)). The year was probably kept from a working-paper version; if it changes, the in-text citation on PDF page 23 changes with it. The inconsistency is internal to the list; the true year is not in the files.
Second check: another model agreed
Report a problem with this finding
Wording "halving the teacher-student ratio also had no effect": the input discussed is more teachers per student (page 2: "lower student-teacher ratios"), so the ratio halved is the student-teacher ratio.
- p. 3 first paragraph
halving the teacher-student ratio also had no
- p. 2 last paragraph
lower student-teacher
Judge's note: Page 3 and page 2 images read. As printed the sentence says teachers per student were halved, the opposite of the intervention (an extra teacher). The meaning is recoverable from context, so it is a wording slip.
Second check: another model agreed
Report a problem with this finding
25 minor (typos and the like; most checked by the judge only)
Reference or label Heckman & Vytlacil 2008a and 2008b are printed with the same title apart from the part number: "Econometrics evaluation of social program part I: using the marginal treatment effect to organize alternative economic estimators to evaluate social programs and to forecast their effect in new environment" and the same for "part II"; the titles also read "Econometrics evaluation of social program".
- p. 27 Literature Cited
Heckman JJ, Vytlacil EJ. 2008a. Econometrics evaluation of social program part I: using
- p. 27 Literature Cited
Heckman JJ, Vytlacil EJ. 2008b. Econometrics evaluation of social program part II: using
- p. 24 Literature Cited (Abbring & Heckman)
Econometrics evaluation of social programs part III
- p. 27 Literature Cited (Heckman & Vytlacil)
Econometrics evaluation of social program part I
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that Part I of this handbook chapter has a different subtitle (on causal models, structural models and econometric policy evaluation) and that the series title is "Econometric evaluation of social programs". Page 27 image confirms the two entries are word for word the same after the part number, with different page ranges (4779–874 and 4875–5144), which suggests one entry was copied from the other. The page 24 entry for Abbring & Heckman has "social programs part III" in the plural, while both entries here have "social program". The true Part I title is not in the files.
Second check: another model agreed
Report a problem with this finding
Wording "Ashraf et al. (2006) worked with microfinance institutions in the Philippines" (plural) is followed by "Gine et al. (2008) worked with the same organization" (singular), which has no single antecedent.
- p. 7 third paragraph
worked with microfinance institu-
- p. 7 third paragraph
worked with the same organization
- p. 7 Section 2
worked with microfinance institutions in the Philippines
- p. 7
Ashraf et al. (2006) worked with microfinance institu-
Judge's note: Page 7 image read. The plural and the singular do not agree; the article does not say how many institutions were involved, so either form could be the slip.
Second check: another model agreed
Report a problem with this finding
Typo Misspelling: "At the falll 2008 BREAD conference" should read "fall".
- p. 2 Introduction, first paragraph
At the falll 2008 BREAD
- p. 2 Intro
falll 2008 BREAD
- p. 6 Sec. 2
particular policy of program
Judge's note: Page 2 image shows three l's; "fall 2008 NEUDC" is spelled correctly three lines above. The "policy of program" item some reviewers attached here is handled in C009.
Second check: none (judge only)
Report a problem with this finding
Typo Author names misspelled in citations: "Imbens & Woolridge (2008)" (PDF pp. 2 and 19; the list and pp. 12-13 have Wooldridge), "Deheijia 2005" (p. 21; list has Dehejia), "Duflo & Chattopdhyay (2004)" (p. 12 footnote; list has Chattopadhyay).
- p. 2 Section 2, first paragraph
Imbens & Woolridge (2008)
- p. 19 Section 3.5, third paragraph
Woolridge (2008) make the case
- p. 27 Literature Cited
Imbens G, Wooldridge JM. 2008.
- p. 19 Section 3.5, third paragraph
Woolridge (2008)
Judge's note: Woolridge seen on the page 2 image and Deheijia on the page 21 image; the reference-list spellings (Wooldridge, Dehejia, Chattopadhyay) and the p. 12 and p. 19 instances were read from the text layer, which is reliable for plain surnames. The Fizbein and Mulainathan items raised here are covered by C005 and C011.
Second check: none (judge only)
Report a problem with this finding
Typo "Fizbein & Schady 2009" (PDF p. 4) and "Fizbein & Schady (2009)" (PDF p. 10) do not match the reference-list entry "Fiszbein A, Schady N, eds. 2009."
- p. 4 last paragraph
Fizbein & Schady 2009
- p. 10 fourth paragraph
see Fizbein & Schady (2009)
- p. 26 Literature Cited
Fiszbein A, Schady N, eds. 2009.
- p. 10 fourth paragraph
Fizbein & Schady (2009)
Judge's note: Text layer shows Fizbein at both in-text citations and Fiszbein in the list (PDF p. 26); a plain surname is not at risk of extraction error. Which spelling is right rests on the reference list, but the two forms differ within the paper.
Second check: none (judge only)
Report a problem with this finding
Typo "PROGRESA-Opportunidades program" (PDF pp. 4 and 10): the programme name is spelled with a double p.
- p. 4 fourth paragraph
PROGRESA-Opportunidades program in Mexico
- p. 10 fourth paragraph
The PROGRESA-Opportunidades program
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that the Mexican programme's name is the Spanish word Oportunidades, with one p. The paper prints "Opportunidades" consistently in both places (text layer) and nothing in the files gives the official spelling.
Second check: none (judge only)
Report a problem with this finding
Typo "which were designed to identify well-designed parameters (i.e., income and substitution effect...)" apparently should read "well-defined parameters"; also "HIV-AIDs" (PDF p. 14) against "HIV-AIDS" (PDF p. 4).
- p. 6 third paragraph
designed to identify well-designed parameters
- p. 6 third paragraph
for a particular policy of program
- p. 14 Sec. 3.1
HIV-AIDs
- p. 4 Sec. 2
HIV-AIDS prevention strategies
Judge's note: Page 6 image read: "well-designed parameters" is printed, and parameters are defined rather than designed (the word echoes "were designed" just before). The HIV-AIDs/HIV-AIDS capitalisation difference was read from the text layer. The "policy of program" item is handled in C009.
Second check: none (judge only)
Report a problem with this finding
Typo PDF p14: "education, malaria, HIV-AIDs, accountability, and transport" - lower-case s in the acronym; PDF p4 prints "HIV-AIDS".
- p. 14 Section 3.1
HIV-AIDs
- p. 4 Section 2
HIV-AIDS prevention
Judge's note: Page image p14 shows "HIV-AIDs" with a lower-case s; the p4 crop shows "HIV-AIDS prevention strategies".
Second check: none (judge only)
Report a problem with this finding
Typo "the average treatment effect for a particular policy of program" should read "policy or program".
- p. 6 third paragraph
for a particular policy of program
Judge's note: Page 6 image shows "policy of program".
Second check: none (judge only)
Report a problem with this finding
Typo "a large impact on presence and tests scores" should read "test scores".
- p. 6 first paragraph
presence and tests scores
Judge's note: Page 6 image shows "tests scores"; the article writes "test scores" elsewhere (for example pages 2 and 3).
Second check: none (judge only)
Report a problem with this finding
Typo Garbled sentence: "The effect the offer price has on keeping the actual price fixed identifies the screening effect"; the object of "has on" is missing (the effect of the offer price on usage, keeping the actual price fixed).
- p. 7 second paragraph
effect the offer price has on keeping the actual price fixed identifies the screening effect
Judge's note: Page 7 image read. The parallel clause that follows ("the variation in the actual price (with a fixed offer price) pins down the sunk cost effect") shows the intended meaning; a word such as "usage" and a comma are missing.
Second check: none (judge only)
Report a problem with this finding
Typo PDF p9: "their evaluation of the Columbia Conditional Cash Transfer Program" spells the country "Columbia"; the paper spells it "Colombia" everywhere else (PDF pp10, 17, 18).
- p. 9 second paragraph
Columbia Conditional Cash
- p. 10 fourth paragraph
(Colombia, Nicaragua, Ecuador, and Honduras;
- p. 9 Section 2, last paragraph
Columbia Conditional Cash Transfer Program
Judge's note: Page image p9 shows "Columbia Conditional Cash Transfer Program" (Attanasio et al. 2008a). Page 10 lists "Colombia" among the countries with a PROGRESA-type conditional cash transfer programme that had a randomized evaluation, and pp17-18 also print "Colombia". No place or institution called Columbia is discussed anywhere in the paper, so the spelling is inconsistent within the paper itself.
Second check: none (judge only)
Report a problem with this finding
Typo PDF p10: "for the results to be anything more than an initial "proof of concept," the program must come from a program that is sufficiently well defined and well understood" - the first "program" appears to be a wrong word (results or evidence).
- p. 10 second paragraph
program must come from a program that is sufficiently well defined
Only the authors can say what is right here.
Judge's note: Page image p10 prints the sentence as quoted. "the program must come from a program" is circular; the clause opens with "for the results to be", so the subject was probably meant to be the results. Only the authors can say what word was intended.
Second check: none (judge only)
Report a problem with this finding
Typo PDF p11: "can be applied here: First, the different site dummies can be treated as covariates in a pooled regression; nonparametric tests of heterogeneity ... can be performed." - "First" is not followed by a "Second".
- p. 11 last paragraph
First, the
Judge's note: Page image p11 and the continuation on p12: after "First," the alternatives are introduced by a semicolon and then "Another way to proceed is"; no "Second" appears. A minor enumeration slip; the meaning is clear.
Second check: none (judge only)
Report a problem with this finding
Typo PDF p12, footnote 3: "Duflo & Chattopdhyay (2004)" - the name is misspelled; the reference list has "Duflo E, Chattopadhyay R. 2004."
- p. 12 footnote 3
Duflo & Chattopdhyay (2004)
- p. 26 Literature Cited
Duflo E, Chattopadhyay R. 2004.
Judge's note: Page image p12 footnote 3 prints "Chattopdhyay"; the reference-list crop on p26 prints "Chattopadhyay". A letter was dropped in the footnote.
Second check: none (judge only)
Report a problem with this finding
Typo PDF p13: "He points out that that, in the JTPA experiment" - duplicated word "that".
- p. 13 third paragraph
points out that that, in the JTPA experiment
- p. 13 Section 3.1, Heckman 1992 paragraph
He points out that that, in the JTPA
Judge's note: Page image p13 shows "points out that that," on a single line, so it is not a line-break artefact.
Second check: none (judge only)
Report a problem with this finding
Typo PDF p17: "Program effects found in a small study may not generalize what will happen when the program is scaled up nationwide" - missing "to" after "generalize".
- p. 17 Section 3.4, first paragraph
may not generalize what will
- p. 17 Section 3.4
may not generalize what will happen
Judge's note: Page image p17, Section 3.4, prints the sentence as quoted, with no "to".
Second check: none (judge only)
Report a problem with this finding
Typo PDF p18: "training program Jovenes in Action in Colombia" - the programme name is printed half in Spanish and half in English.
- p. 18 Section 3.4
Jovenes in Action in Colombia
- p. 18
training program Jovenes in Action in Colombia
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: the programme's name is Jóvenes en Acción. Page image p18 prints "Jovenes in Action" in italics. The mix of languages is visible on the page; the proper name itself cannot be verified from the files.
Second check: none (judge only)
Report a problem with this finding
Typo PDF p20: "cannot possibly be counted in favor of methods not based on random assignment" - the paragraph ends without a full stop.
- p. 20 third paragraph, last line
not based on random assignment
Judge's note: Page image p20: the line ends at "not based on random assignment" with no period, and the next paragraph begins "Moreover, a lot of the heterogeneity".
Second check: none (judge only)
Report a problem with this finding
Typo PDF p20: "experiments still have the advantage over methods that, with few assumptions, one can know important aspects of impact of the treatment" - the sentence is ungrammatical as printed; a word such as "other" appears to be missing after "over".
- p. 20 third paragraph
the advantage over methods that, with few assumptions
Only the authors can say what is right here.
Judge's note: Page image p20 prints the sentence as quoted. As written, "methods that ... one can know" does not parse; the likely intent is "the advantage over other methods that, with few assumptions, one can know ...". The authors should say what was intended.
Second check: none (judge only)
Report a problem with this finding
Typo In-text citation "Deheijia 2005" misspells the author; the reference list has "Dehejia R. 2005. Program evaluation as a decision problem."
- p. 21 third paragraph
Deheijia 2005
- p. 26 Literature Cited
Dehejia R. 2005.
- p. 26 Literature Cited
Dehejia R. 2005. Program evaluation as a decision problem.
Judge's note: Page 21 image shows "Manski 2000, 2002, 2004; Deheijia 2005; Hirano & Porter 2005". Page 26 image shows the entry as "Dehejia R. 2005". The spelling in the text has an extra i.
Second check: none (judge only)
Report a problem with this finding
Typo Reference entry "Bertrand M, Karlan D, Mulainathan S, Zinman J. 2009." misspells Mullainathan.
- p. 25 Literature Cited
Bertrand M, Karlan D, Mulainathan S, Zinman J. 2009.
- p. 25 Literature cited
Mulainathan S, Zinman J. 2009.
Judge's note: Page 25 image: the two entries directly above print "Mullainathan S" (Bertrand, Chugh & Mullainathan 2005; Bertrand, Djankov, Hanna & Mullainathan 2007), and this entry prints "Mulainathan S" with one l.
Second check: none (judge only)
Report a problem with this finding
Typo Reference title is cut short: "What's advertising content worth? Evidence from a consumer credit marketing." ends without its noun.
- p. 25 Literature Cited, Bertrand et al. 2009
from a consumer credit marketing.
Judge's note: Page 25 image shows the title ending at "from a consumer credit marketing." followed by the journal. The phrase "a consumer credit marketing" is grammatically incomplete, so words are missing (the list's Karlan & Zinman 2005 entry has the parallel phrase "with a consumer credit field experiment"). The exact missing words rest on the published title, which is not in the files.
Second check: none (judge only)
Report a problem with this finding
Typo Reference list out of alphabetical order: "Heckman JJ, Smith J, Clements N. 1997b." is placed before "Heckman JJ, Schmierer D, Urzua S. 2010."
- p. 27 Literature Cited
Heckman JJ, Smith J, Clements N. 1997b.
Judge's note: Page 27 image: the Heckman entries run by second author (Ichimura, Ichimura, Leamers, Lochner, Smith, Schmierer, Urzua, Vytlacil). Smith comes before Schmierer, which is out of order; the list is not chronological either (1998 precedes 1997a).
Second check: none (judge only)
Report a problem with this finding
Typo Reference list out of alphabetical order: "Topalova P, Duflo E. 2003." is placed before "Thornton R. 2007." and "Todd P, Wolpin KI. 2006."
- p. 28 Literature Cited
Topalova P, Duflo E. 2003.
Judge's note: Page 28 image shows the order Rubin, Topalova, Thornton, Todd, World Bank. Topalova should follow Todd.
Second check: none (judge only)