Estimating the Reproducibility of Psychological Science
Science, 2015 · paper · Psychology
Reports replications of 100 published psychology studies using high-powered designs. Replication effects were about half the original size; 97% of originals but only 36% of replications were significant, and success was better predicted by original evidence strength.
Checked: PDF from pure.ed.ac.uk, fetched 5 Oct 2026. Page numbers are those of that PDF.
16 findings
Substance Two different per-journal sizes are given for the pool available for selection. p.8 has 64 PSCI, 55 JPSP and 39 JEP:LMC, totalling 158 eligible articles. p.10 says "made available for selection: JPSP (n=59 articles), JEP:LMC (n=40 articles), and PSCI (n=68 articles)", which totals 167.
- p. 8 Method, sampling frame paragraph
39 of 64 (61%) articles from PSCI, 31 of 55 (56%) articles from JPSP, and 28 of 39 (72%) articles from JEP:LMC
- p. 10 Publishing journal and subdiscipline
JPSP (n=59 articles), JEP:LMC (n=40 articles), and PSCI (n=68 articles)
- p. 10 Publishing journal and subdiscipline
JPSP (n=32 studies), JEP:LMC (n=28 studies), and PSCI (n=40 studies)
Only the authors can say what is right here.
Judge's note: On the p.8 image, 158 eligible articles are split 64/55/39. This agrees with 111 selected + 47 not claimed = 158. On the p.10 image, the pool 'made available for selection' is 59/40/68 = 167. I searched the text for 167 and for 'available' and found no definition that separates the two pools. The third reviewer's 98-vs-100 point (articles vs studies) is explained on p.8, 'Two articles had two replications each', and on p.52, so that part is not a defect. The remaining discrepancy is 158 vs 167, and only the authors can say which figure is right.
Second check: another model agreed
Report a problem with this finding
Substance The supplement says "94 (75+19) could be used to determine if the CI of the replication contained the effect size". The A3 table total (95), the Table S4 notes ("5 excluded, n=95") and the coverage analysis (73 + 18 + 4 = 95; 47.4%) all give 95.
- p. 53 Preliminary analyses
94 (75+19) could be used to determine if the CI of the replication contained the effect size
- p. 75 Appendix A3 table, total row
95
- p. 12 Table 1 note
(5 excluded, n = 95 studies)
- p. 56 Supplement coverage analysis
subset of 18 and 4 studies
Judge's note: PDF p53 (printed 52) prints '94 (75+19)'. In the PDF p75 A3 table, the '% original effect size within replication 95% CI' column marks rows 69+18+2+2+4 = 95, with a total of 95. The PDF p67 notes give n=95. On PDF p56, 30/73 plus 68.2% of 22 gives 47.4% of 95. No alternative grouping yielding 94 was found.
Second check: another model agreed
Report a problem with this finding
Substance "26 of 63 (41%) ... 6 of 23 (26%) ... 2 of 11 (18%)": the denominators sum to 97, but the successes sum to 34, not the 35 of 97 reported in Table 1 and the Results.
- p. 22 Correlates of Reproducibility
26 of 63 (41%) original studies with P < 0.02 achieved P < 0.05 in the replication, whereas 6 of 23 (26%)
- p. 22 same paragraph
2 of 11 (18%) that had a P value > 0.04 did so
- p. 22 Correlates of Reproducibility
26 of 63 (41%) original studies with P < 0.02 achieved P < 0.05
- p. 22 Correlates of Reproducibility
26 of 63 (41%) original studies with P < 0.02 achieved P < 0.05 in the replication, whereas 6 of 23 (26%) that had a P value between 0.02 < P < 0.04 and 2 of 11 (18%)
Only the authors can say what is right here.
Judge's note: I read the p.22 image. The three P-value bands cover 63+23+11 = 97 studies, which is the full set of significant originals. The successes in the bands add to 34, whereas Table 1 (p.12) and the Results (p.17) report 35 of 97. I found no text that removes a study from these bands. Each percentage matches its own fraction, so the authors would need to say which count is wrong.
Second check: another model agreed
Report a problem with this finding
Substance The methods code highest degree as "4 = PhD/equivalent, 3 = Master's/equivalent, 2 = some graduate school, 1 = Bachelor's/equivalent" and 72 of 85 senior members held a PhD. Yet Table S4 reports M = 1.24, median = 1 for "Highest degree of senior member". These values fit only a reversed coding (1 = PhD), which also reverses how the sign of the correlations should be read.
- p. 49 Highest degree of replication team’s senior member
4 = PhD/equivalent, 3 = Master’s/equivalent, 2 = some graduate school, 1 = Bachelor’s/equivalent
- p. 66 Table S4, Highest degree of senior member
1.24 0.62 1 1 to 4
- p. 44 Replication Teams
72 had a PhD or equivalent
- p. 66 Table S4
of senior member 1.24 0.62 1 1 to 4
Only the authors can say what is right here.
Judge's note: PDF p49 (printed 48) gives the coding. PDF p44 gives the counts 72/9/1/3. PDF p66 Table S4 shows 1.24, 0.62, 1, 1 to 4. Under the stated coding the mean is about 3.76. Under reversed coding it is 1.235, which matches the printed 1.24. Counting replications (100) rather than unique members cannot bring the median down from 4 to 1. No statement of reverse coding was found.
Second check: another model agreed
Report a problem with this finding
Substance The methods code position as "7 = Professor (or equivalent) ... 1 = Undergraduate student" and 62 of 85 senior members were faculty (codes 5-7). Yet Table S4 gives "Position of senior member of replication team" M = 2.91, median = 2. This is consistent only with a reversed coding.
- p. 49 Position of senior member of replication team
7 = Professor (or equivalent), 6 = Associate Professor (or equivalent), 5 = Assistant Professor
- p. 66 Table S4, Position of senior member
2.91 1.89 2 1 to 7
- p. 44 Replication Teams
62 were faculty members or equivalent
Only the authors can say what is right here.
Judge's note: PDF p49 gives the coding. PDF p44: 62 faculty, 8 post-docs, 13 graduate students, 1 undergraduate, 1 private sector. PDF p66 Table S4: 2.91, 1.89, 2, 1 to 7. With at least 62% faculty (62 of 85 members, and at least 62 of the 100 replications), the median under the stated coding must be 5 or higher. A median of 2 ('Master's student') is impossible. The direction of the correlations in Table S4 is therefore ambiguous. No statement of reverse coding was found.
Second check: another model agreed
Report a problem with this finding
Substance Appendix A7: standard errors could be computed for r, t or F(1,df2) "which was for 74 study-pairs", but subset MA is 73 pairs everywhere else ("Seventy-three study-pairs could be included in subset MA"; A3 table total 73).
- p. 84 Appendix A7, first paragraph
F(1,df2), which was for 74 study-pairs
- p. 75 Appendix A3 table total row, Meta-analytic estimate (subset MA)
73
- p. 84 A7 first paragraph
which was for 74 studypairs
- p. 75 A3 table
73 75 95
Judge's note: p84 image shows '74 study-pairs' in the sentence defining subset MA. p53 crop shows 73 in subset MA and 75 (73+2) for the significance test. The p75 A3 table total row shows 73 for the MA column, and the Table S2/S4 notes give n = 73. 74 matches neither 73 nor 75.
Second check: another model agreed
Report a problem with this finding
Substance "A positive effect of standard error on effect size indicates that studies' effect sizes are positively associated with their sample sizes". Because standard error falls as sample size rises, a positive SE effect means a negative association with sample size.
- p. 58 Meta-analysis of all original study effects
A positive effect of standard error on effect size indicates that studies’ effect sizes are positively associated with their sample sizes.
- p. 58 same paragraph
using larger sample sizes in fields where smaller effect sizes are observed
Judge's note: p58 image: the sentence is as quoted. SE is proportional to 1/sqrt(N), so a positive SE coefficient means smaller studies show larger effects. The same paragraph ('larger sample sizes in fields where smaller effect sizes are observed') and p60 ('larger replication samples were used for smaller true effects') assume the negative relation. The stated direction is reversed.
Second check: another model agreed
Report a problem with this finding
Substance In Table S4 the Replication df/N row reads "4804 4574 68.5 7 to 455304". An SD of 4574 is arithmetically impossible with mean 4804, maximum 455304 and about 100 studies.
- p. 67 Table S4, Replication df/N row
4804 4574 68.5 455304
Only the authors can say what is right here.
Judge's note: p67 image confirms the values. With n=100, the single maximum alone gives SD of at least (455304-4804)/sqrt(99), about 45,277. The analogous Table S3 original row (mean 2409, SD 22994, max 230025) is consistent: (230025-2409)/sqrt(99) is about 22,876. The SD is probably missing a digit (e.g. 45740), but only the authors can confirm.
Second check: another model agreed
Report a problem with this finding
Substance Appendix A5 prints the rearranged estimator as λ̂ = (df2 − 2) × F × df1 / df2, omitting the "− df1" term that follows from the stated expected value and that the code subtracts.
- p. 77 Appendix A5, equation for λ̂
Rewriting this expected value yields
- p. 82 A5 code
ncp.o[i] <- F.o*df1.o*(df2.o-2)/df2.o-df1.o
Only the authors can say what is right here.
Judge's note: p77 image shows E[F] = (df1+λ)/df1 × df2/(df2−2). Solving: F·df1·(df2−2)/df2 = df1 + λ, so λ = F·df1·(df2−2)/df2 − df1. The printed equation lacks −df1. The code (p82) uses ncp.o[i] <- F.o*df1.o*(df2.o-2)/df2.o-df1.o, so the computation is unaffected, but the formula as typeset is wrong.
Second check: another model agreed
Report a problem with this finding
Reference or label "narrow-and-deep approach is characterized by the Many Labs replication projects (34)" cites ref 34, which is Sterling, "Publication decisions and their possible effects on inferences". The Many Labs project is ref 32 (Klein et al.).
- p. 27 Implications and Limitations
characterized by the Many Labs replication projects (34)
- p. 32 Reference 34
34. T. D. Sterling, Publication decisions and their possible effects on inferences
- p. 27 Discussion, Implications and Limitations, first line
narrow-and-deep approach is characterized by the Many Labs replication projects (34)
- p. 32 Reference list
32. R. Klein, et al., Investigating variation in replicability: A “many labs” replication project.
Judge's note: The p.27 image cites (34). In the reference list on the p.32 image, 34 is Sterling and 32 is Klein et al., 'Investigating variation in replicability: A "many labs" replication project'. The citation points to the wrong entry.
Second check: another model agreed
Report a problem with this finding
Reference or label Reference 34 is truncated: "34. T. D. Sterling, Publication decisions and their possible effects on inferences". It has no journal, volume, pages or year. Reference 11 also lacks its closing parenthesis: "10, 712-713 (2011".
- p. 32 Reference list
34. T. D. Sterling, Publication decisions and their possible effects on inferences
- p. 30 Reference 11
10, 712713 (2011
- p. 32 Reference 41
Psychol. Sci. 25, 729 (2013)
Judge's note: PDF p32 (printed 31) shows reference 34 ending mid-title with no bibliographic details, so the source cannot be identified from the list. PDF p30 shows reference 11 ending '(2011' with no closing parenthesis. The point about the year of reference 41 is judged separately under C018.
Second check: another model agreed
Report a problem with this finding
Reference or label "Results are reported in Table S5" (replication team surprised by outcome), but the supplement contains only Tables S1-S4. The variable appears in the last row of Table S4.
- p. 51 Other variables: Replication team surprised by outcome
Results are reported in Table S5.
- p. 67 Table S4 last row
Replication team surprised by outcome of replication
- p. 33 Supplementary materials list
Tables S1-S4
- p. 67 Table S4 last row
team surprised by outcome of replication
Judge's note: PDF p51 contains the pointer. PDF p67 shows the 'Replication team surprised by outcome of replication' row at the end of Table S4. The supplementary list on PDF p33 reads 'Tables S1-S4'. The moderator-analysis introduction (text p63) is similarly off by one: it cites Tables S3/S4/S5 for what are captioned Tables S2/S3/S4.
Second check: another model agreed
Report a problem with this finding
Reference or label "The density and cumulative P value distributions ... are presented in Figures S1 and S2 respectively", but Figure S1 is captioned "Cumulative P value distributions" and Figure S2 "Density P value distributions".
- p. 53 Evaluating replication effect against null
The density and cumulative P value distributions of original and replication studies are presented in Figures S1 and S2 respectively.
- p. 54 Figure S1 caption
Figure S1: Cumulative P value distributions
- p. 55 Figure S2 caption
Figure S2: Density P value distributions
- p. 54 Figure S1 caption
Figure S1: Cumulative P value distributions of original and replication studies.
Judge's note: The text is on PDF p53. PDF p54 shows Figure S1 titled 'P-value distributions (CDF)'. PDF p55 shows Figure S2 titled 'P-value distributions (PDF)'. 'Respectively' maps them the wrong way round.
Second check: another model agreed
Report a problem with this finding
Reference or label The Moderator Analyses intro cites "(Table S3)" for correlations among indicators, "(Table S4)" for original-study characteristics and "(Table S5)" for replication characteristics, but the tables are labelled Table S2, S3 and S4.
- p. 63 Moderator Analyses intro
correlations among the five reproducibility indicators (Table S3)
- p. 63 Table title
Table S2. Spearman’s rank order correlations among reproducibility indicators
- p. 63 Moderator Analyses intro
reproducibility indicators (Table S3), correlations of individual characteristics of original studies with reproducibility indicators (Table S4)
- p. 63 Table caption
Table S2. Spearman’s rank order correlations among reproducibility indicators
Judge's note: p63 image: the caption directly below the text reads 'Table S2. Spearman's rank order correlations among reproducibility indicators'. Per the cluster's crops, the captions on p64 and p66 cover original and replication characteristics (S3 and S4). No Table S5 exists. All three pointers are off by one.
Second check: another model agreed
Report a problem with this finding
Reference or label Appendix A6 says "The code for calculating the observed proportion can be found in supplement [A6]", but A6 has no such code. It is at the end of A7 ('How often is original study within CI of replication').
- p. 83 Appendix A6
The code for calculating the observed proportion can be found in supplement [A6].
- p. 85 Appendix A7 code
How often is original study within CI of replication
- p. 85 A7 code
sum(in.ci)/length(in.ci) # Proportion of times the original study is within the CI of the replication
Judge's note: p83 image: A6 contains only the t.test, wilcox.test and binom.test calls. The p85 image shows the in.ci coverage code under A7. The pointer refers to its own section and should read A7.
Second check: another model agreed
Report a problem with this finding
Wording Appendix A4: "then this probability is a function of both studies' effect size". With equal true effects assumed, coverage depends on the two studies' sample sizes.
- p. 76 Appendix A4, first paragraph
then this probability is a function of both studies’ effect size
Judge's note: p76 image: the next sentences describe equal sample size (83.4%) and much larger sample sizes, and the code uses only N.r and N.o. The sentence misnames the determining variable.
Second check: another model agreed
Report a problem with this finding
19 minor (typos and the like; most checked by the judge only)
Reference or label Figure S1, titled "P-value distributions (CDF)", labels its y-axis "Density" although it plots cumulative proportions from 0 to 1.
- p. 53 Figure S1 y-axis
Density
Judge's note: The figure is on PDF p54 (the cluster cites p53). The y-axis label 'Density' is visible on the cumulative plot. Figure S3's CDF panel uses 'Cumulative density' instead.
Second check: another model agreed
Report a problem with this finding
Wording The Limitations say "in each article, only one study was replicated", but the supplement says "experiment 7 of Albarracín et al. (2008) was replicated in a lab setting and a secondary replication of experiment 5 was conducted online", and counts these replications among the 100 completed.
- p. 26 Discussion, Implications and Limitations
in each article, only one study was replicated
- p. 52 Supplement, Guide to the Information Commons
experiment 7 of Albarracín et al. (2008) was replicated in a lab setting and a secondary replication of experiment 5 was conducted online
- p. 8 Method, study selection
Two articles had two replications each
Only the authors can say what is right here.
Judge's note: The p.26 image has the blanket limitation statement. The p.52 image states that two different experiments of ref 40 were replicated and that the supplementary replications 'bring ... total completed to 100'. The per-journal counts of 98 articles and 100 studies agree with this. The limitation statement therefore overstates the case for at least one article.
Second check: another model agreed
Report a problem with this finding
Wording p.10 says potential moderators included "five characteristics of the replication", but p.16 lists "seven indicators of the replication study" and Table 2 has seven replication rows.
- p. 10 Measures and Moderators
six characteristics of the original study, and five characteristics of the replication
- p. 16 Analysis of moderators
seven indicators of the replication study
- p. 10 Measures and Moderators
five characteristics of the replication
- p. 14 Table 2, Replication Characteristics
Self-assessed quality of replication
Only the authors can say what is right here.
Judge's note: p.10 says six original and five replication characteristics. p.16–17 lists six original and seven replication indicators (P value, effect size, power, sample size, challenge, expertise, quality). The Table 2 image on p.14 has seven rows under Replication Characteristics. The 'five' might exclude replication P value and effect size, but the paper never says so.
Second check: another model agreed
Report a problem with this finding
Wording The coverage test is described as a "goodness-of-fit χ2 test" (p.15 and A6), but A6 says the inputs "was entered in the binomial test" and gives "binom.test(f, n, p, "two.sided", 0.95)".
- p. 15 Effect sizes (Methods)
compared this with the expected proportion using a goodness-of-fit χ2 test
- p. 83 Appendix A6
binom.test(f, n, p, "two.sided", 0.95)
Only the authors can say what is right here.
Judge's note: Both the p.15 image and the p.83 A6 image name a goodness-of-fit χ² test for the coverage comparison. The same A6 paragraph then gives R code for an exact binomial test. Two procedures are named for the same test, so the authors should say which one was used.
Second check: another model agreed
Report a problem with this finding
Wording The "Replication team method expertise" paragraph says "Replication teams self-rated the domain expertise of the senior member of the project", but its scale anchors refer to "the methodology".
- p. 49 Replication team method expertise
Replication teams self-rated the domain expertise of the senior member of the project
Judge's note: PDF p49 (printed 48): the method-expertise paragraph repeats 'domain expertise' from the preceding paragraph, while all its anchors concern methodology. This is a copy-paste slip.
Second check: another model agreed
Report a problem with this finding
Wording The Effect similarity frequencies "(12), (16), (8), (12), (14), (9), (28)" sum to 99, not 100, and no missing response is mentioned. M = 3.60 matches n = 99.
- p. 51 Effect similarity
7 = virtually identical (12), 6 = extremely similar (16), 5 = very similar (8), 4 = moderately similar (12), 3 = somewhat similar (14), 2 = slightly similar (9), 1 = not at all similar (28)
Judge's note: PDF p51 (printed 50). The weighted sum is 356, and 356/99 = 3.596, which rounds to the printed 3.60, so the figures are internally consistent with one team not rating. The parallel Findings-similarity counts sum to 100. The paper never states that n = 99 for this item.
Second check: another model agreed
Report a problem with this finding
Wording "average effect size was smaller for JPSP ... than for the other four journal/discipline categories", but the paper defines only four categories (JPSP-social, JEP:LMC-cognitive, PSCI-social, PSCI-cognitive), so JPSP has three others.
- p. 58 Combining original and replication effect sizes
than for the other four journal/discipline categories
- p. 58 Meta-analysis section
Discipline is a categorical variable with categories JPSP-social (= reference category), JEP:LMC-cognitive, PSCI-social, and PSCI-cognitive.
Judge's note: p58 image: the discipline variable is defined with four categories, the reference being JPSP-social. Table 1 has an Overall column plus four category columns, so "four" counts the Overall column, which is not a journal/discipline category. The comparison itself (.138 lowest) holds against the three others.
Second check: another model agreed
Report a problem with this finding
Typo "Abstract word count = 149 words", but the printed abstract has about 139 words.
- p. 3 Below abstract
Abstract word count = 149 words
Judge's note: I counted the abstract on the p.3 image sentence by sentence and got 18+22+13+8+7+16+12+19+24 = 139 words. This is an administrative submission line, probably left over from an earlier draft. It is cosmetic.
Second check: none (judge only)
Report a problem with this finding
Typo "obtaining false—positive results" uses an em dash where the hyphen in 'false-positive' belongs.
- p. 5 Introduction, paragraph 2
obtaining false—positive results in particular
Judge's note: The crop of p.5 shows a long em dash between 'false' and 'positive'. The image confirms it, so it is not an extraction artefact.
Second check: none (judge only)
Report a problem with this finding
Typo In Table 1, the PSCI-Cognitive original effect size is printed ".53 (.2)"; every other SD has two or three decimals.
- p. 12 Table 1, original effect size, PSCI-Cognitive
.53 (.2)
Judge's note: The Table 1 image on p.12 shows '.53 (.2)', while the other cells read '(.10)', '(.18)', '(.20)'. This is an inconsistency in precision only.
Second check: none (judge only)
Report a problem with this finding
Typo The Fig. 2 legend uses "JEPLMC", "JPSP" and "PS", while the text and Table 1 use JEP:LMC and PSCI.
- p. 19 Fig. 2 legend
JEPLMC / JPSP / PS
- p. 12 Table 1 column headers
JEP:LMC - Cognitive
Judge's note: The p.19 legend reads JEPLMC / JPSP / PS; the Table 1 headers on p.12 read 'JEP:LMC - Cognitive' and 'PSCI'. The abbreviations are inconsistent, but the meaning is clear.
Second check: none (judge only)
Report a problem with this finding
Typo Overall combined meta-analytic mean correlation printed as ".310 (SD = 0.223)" in the supplement (p58) but ".309 (.223)" in Table 1 (p12).
- p. 58 Combining original and replication effect sizes
was .310 (SD = 0.223)
- p. 12 Table 1, Meta-analytic Mean (SD) Estimate, Overall
.309
Judge's note: Both images checked. Same quantity (75 pairs, same SD .223) printed with different third decimals. Probably a rounding difference (e.g. .3095), so this is a minor consistency point.
Second check: none (judge only)
Report a problem with this finding
Typo "[M = 0.92, median (Mdn) = 0.95)" opens with a square bracket and closes with a parenthesis.
- p. 17 Results, first paragraph
[M = 0.92, median (Mdn) = 0.95)
- p. 17 Results, first paragraph
[M = 0.92, median (Mdn) = 0.95), we would expect
Judge's note: The p.17 image shows '[M = 0.92, median (Mdn) = 0.95),' so the brackets do not match.
Second check: none (judge only)
Report a problem with this finding
Typo The Fig. 2 caption says "Dotted red lines indicate P = 0.05 criterion", but the dashed P = 0.05 lines in the figure are dark grey/black.
- p. 19 Fig. 2 caption
Dotted red lines indicate P = 0.05 criterion.
Judge's note: On the p.19 image, the vertical and horizontal dashed lines at 0.05 in the main panel and the subplot are dark grey, not red. The grey connector lines that mark the 0 to 0.005 range are present. The colour named in the caption does not match the figure.
Second check: none (judge only)
Report a problem with this finding
Typo Reference 11 ends "Nat. Rev. Drug Disc. 10, 712-713 (2011" with no closing parenthesis or final period.
- p. 30 Reference 11
10, 712-713 (2011
- p. 30 Reference list, entry 11
Nat. Rev. Drug Disc. 10, 712-713 (2011
Judge's note: The p.30 image shows reference 11 ending at '(2011' with no closing parenthesis or full stop.
Second check: none (judge only)
Report a problem with this finding
Typo Reference 26 abbreviates the journal as "Math. Pro. Camb. Phil. Soc." where the usual form is "Math. Proc."
- p. 31 Reference 26
Math. Pro. Camb. Phil. Soc.
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: the standard abbreviation of the journal title (Proceedings → 'Proc.'). The crop of reference 26 on PDF p31 shows 'Math. Pro. Camb. Phil. Soc. 22, 700-725 (1925)'. This is cosmetic.
Second check: none (judge only)
Report a problem with this finding
Typo The affiliation list contains three misspellings: "University of Illinios at Chicago" (102), "Worchester Polytechnic Institute" (118) and "Centre for Resarch in Psychology, Behaviour and Achievement" (121).
- p. 37 Affiliations 102, 118, 121
University of Illinios at Chicago
- p. 37 Affiliation 118
Worchester Polytechnic Institute
- p. 37 Affiliation 121
Centre for Resarch in
- p. 37 Affiliations
Centre for Resarch in Psychology
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: the correct spellings of the institution names (Illinois, Worcester). 'Resarch' is a plain typo. All three spellings are visible in the crops of PDF p37.
Second check: none (judge only)
Report a problem with this finding
Typo The running page numbers go 53 (PDF p54), 54 (PDF p55), 56 (PDF p56), so printed page 55 is skipped.
- p. 55 Running header of Figure S2 page
54
- p. 56 Running header
56
- p. 56 running header
Reproducibility Project 56
- p. 56 Figure S3 caption
Distributions (left) and cumulative distribution functions of effect sizes of original and replication studies.
Judge's note: I checked the headers on PDF pp54-56. The Figure S3 caption's lack of '(right)' is minor, and the body text on the same page does say '(right)'. All of this is cosmetic.
Second check: none (judge only)
Report a problem with this finding
Typo The Table S2 footnote repeats the label: "Notes: Effect size difference ... Fisher's z. Notes: Four original results ...".
- p. 63 Table S2 notes
Notes: Effect size difference (original - replication) computed after converting r's to Fisher's z. Notes: Four
- p. 63 Table S2 notes
"meta-analytic mean estimates" (27 excluded; n = 73)
Judge's note: p63 image shows 'Notes:' twice in the same footnote. The Table S4 note on p67 uses a single 'Notes:'. Cosmetic.
Second check: none (judge only)