Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement
The Lancet, 1986 · paper · Biomedicine
Examines how to compare a new clinical measurement method with an established one. It argues that correlation coefficients are misleading for judging agreement and describes a graphical alternative with simple calculations, illustrated with peak expiratory flow meter data, plus its link to repeatability.
Checked: PDF from www-users.york.ac.uk, fetched 4 Oct 2026. Page numbers are those of that PDF.
4 findings
Substance The upper limit of agreement is printed with a minus sign: "d + 2s = −2.1 + (2×38.8) = −75.5 l/min"; the sum is +75.5.
- p. 4 Measuring agreement, second displayed equation
−2.1 + (2×38.8) = −75.5 l/min
- p. 4 sentence after the equations
76 l/min above
- p. 5 Precision of estimated limits, first paragraph
For the upper limit of agreement the 95% confidence interval is 40.9 to 110.1
- p. 4 Measuring agreement, second displayed equation
−2.1 + (2×38.8) = −75.5
Judge's note: Page 4 image shows "=−75.5 l/min" on the d+2s line, so the minus sign is printed. The sum is +75.5. The paper uses the positive value elsewhere: "76 l/min above" in the next sentence, the Mean + 2SD line in fig 2 near +75, and the interval "40.9 to 110.1" on page 5, which is 75.5 ± 2.12×16.3.
Second check: another model agreed
Report a problem with this finding
Reference or label "a logarithmic transformation should yield a picture more like that of figs 2 and 4" cites fig 4, the plot whose scatter increases with the mean; figs 2 and 3 are evidently meant.
- p. 5 Relation between difference and mean, last sentence
a logarithmic transformation should yield a picture more like that of figs 2 and 4
- p. 5 Relation between difference and mean
Fig 4 compares the measurement of
- p. 5 Relation between difference and mean
a picture more like that of figs 2 and 4
Judge's note: Page 5 image shows the sentence. The same paragraph introduces fig 4 as the case where "The scatter of the differences increases as the VCF increases", and fig 5 is its log-transformed version (page 6). The plots showing no relation between difference and mean are fig 2 (page 4) and fig 3 (page 5).
Second check: another model agreed
Report a problem with this finding
Reference or label Reference 7 title reads "Precision of test methods 1: Guide for the determination and reproducibility for a standard test method (BS 597, Part 1). London: BSI (1975).": words appear to be missing after "determination" (probably "of repeatability"), and the standard number and year may be misprinted.
- p. 9 Reference 7
Guide for the determination and reproducibility for a standard test method (BS 597, Part 1). London: BSI (1975).
- p. 9 Reference 7
Guide for the determination and reproducibility for a standard test method (BS 597, Part 1)
- p. 7 Repeatability, second paragraph
coefficient adopted by the British Standards Institution. 7
- p. 9 Reference 7
Guide for the determination and reproducibility for a standard test method
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that the standard is BS 5497 Part 1, dated 1979, titled "Guide for the determination of repeatability and reproducibility for a standard test method". None of this can be checked from the files. What the page 9 image does show: the title as printed, "Guide for the determination and reproducibility for a standard test method", does not parse because "determination" has no object. The page 7 crop shows reference 7 is cited for "the definition of a repeatability coefficient adopted by the British Standards Institution", yet the word "repeatability" is absent from the printed title, which supports the reading that "of repeatability" was dropped. The year is also the only one in the list set in parentheses. The garbled title is evident on the page; the number "BS 597" and year "(1975)" are a question for the authors.
Second check: another model agreed
Report a problem with this finding
Wording "Thus, the mini meter may be 80 l/min below or 76 l/min above the large meter" reverses the directions: the difference is defined as "large meter minus small meter", so the limits −79.7 and +75.5 mean the mini meter may be 80 above or 76 below the large meter.
- p. 4 Measuring agreement, sentence after the equations
Thus, the mini meter may be 80 l/min below or 76 l/min above the large meter
- p. 4 Measuring agreement, first paragraph
the mean difference (large meter minus small meter) is
- p. 4 Fig 2, y-axis label
(large - mini)
- p. 4 Limits of agreement, after calculation
the mini meter may be 80 l/min below or 76 l/min above the large meter
Judge's note: Page 4 image: the text defines the difference as large minus small and the fig 2 axis reads "(large - mini)". The first-measurement columns of the table (page 2 image) give a sum of large − mini of −36 over 17 subjects, mean −2.1, so that convention is the one used. The largest negative difference is subject 15 (178 − 259 = −81, mini higher) and the largest positive is subject 16 (423 − 350 = 73, mini lower). The two magnitudes are close, so the practical message is unchanged; rated S3. The "Differene" axis-label spelling also listed in this cluster is handled under C006.
Second check: another model agreed
Report a problem with this finding
12 minor (typos and the like; most checked by the judge only)
Wording "the short axis measurement of VCF will be between 0.80 and 1.27 times the long axis VCF" gives the ratio as short/long, while the axes of figs 4 and 5 label the difference "(long-short)", under which the antilogs 0.80 and 1.27 bound long/short.
- p. 6 paragraph under Fig 4
the short axis measurement of VCF will be between 0.80 and 1.27 times the long axis VCF
- p. 6 paragraph under Fig 4
differ from the long axis measurement by 20% below to 27% above
- p. 6 Fig 5, y-axis label
Difference in log VCF (long-short)
- p. 6 Fig 5 paragraph
we get 0.80 and 1.27
Only the authors can say what is right here.
Judge's note: Page 6 image: both y-axes read "(long-short)", and the text interprets the antilogs of −0.098 and 0.106 as short relative to long. Either the axis labels or the sentence has the direction reversed; only the authors can say which. If the labels are right, short/long would run from 1/1.27 to 1/0.80, about 0.79 to 1.25. The secondary point (antilog of 0.106 is 1.276, printed 1.27) is not confirmed: the limit is itself rounded to three decimals, and a value just under 0.106 gives 1.27.
Second check: another model agreed
Report a problem with this finding
Typo The introduction cites "a set of data (Table 1)" but the only table is headed "PEFR MEASURED WITH WRIGHT PEAK FLOW AND MINI WRIGHT PEAK FLOW METER" with no number, and is later called "The table" (page 7) and "the table" (page 8).
- p. 1 Introduction, last paragraph
a set of data (Table 1)
- p. 2 Table title
PEFR MEASURED WITH WRIGHT PEAK FLOW AND MINI WRIGHT PEAK FLOW METER
- p. 7 Repeatability, second paragraph
The table shows paired data for PEFR
- p. 7 Repeatability, second paragraph
The table shows paired data for PEFR.
Judge's note: Page 2 image shows the table title without a number; page 7 image shows "The table shows paired data for PEFR"; page 8 image shows "as in the table". The "(Table 1)" wording on page 1 was read from the text layer only. There is one table, so the target is unambiguous; this is a labelling inconsistency.
Second check: none (judge only)
Report a problem with this finding
Typo Table column header for the Wright meter Second PEFR reads "(l/mi)"; the other three columns read "(l/min)".
- p. 2 Table, header of the Wright meter Second PEFR column
(l/mi)
- p. 2
Subject (l/min) (l/mi)
Judge's note: Page 2 image shows "(l/mi)" under Second PEFR for the Wright peak flow meter and "(l/min)" in the other three columns. Not an extraction artefact.
Second check: none (judge only)
Report a problem with this finding
Typo Comma splice: "discrepancies of up to 80 l/min, these differences are not obvious from fig 1".
- p. 3 Measuring agreement, paragraph
discrepancies of up to 80 l/min, these differences are not obvious from fig 1
Judge's note: Read from the text layer of page 3 (plain prose, no numbers at issue); the page image was not opened. Two independent clauses are joined by a comma. Punctuation only; the meaning is clear. The reviewer's title refers to a different sentence ("worse below ... and worse above"), which is not a defect.
Second check: none (judge only)
Report a problem with this finding
Typo Y-axis labels of fig 2 ("Differene in PEFR (large - mini) (l/min)") and fig 6 ("Differene in PEFR by mini meter (l/m") misspell "Difference"; the fig 6 label is also cut off at "(l/m".
- p. 4 Fig 2, y-axis label
Differene in PEFR (large - mini) (l/min)
- p. 7 Fig 6, y-axis label
Differene in PEFR by mini meter (l/m
- p. 7 Fig 6 y-axis label
Differene in PEFR by mini meter
Judge's note: Both labels read directly from the page 4 and page 7 images. Figs 3, 4 and 5 (pages 5 and 6) spell "Difference" correctly.
Second check: none (judge only)
Report a problem with this finding
Typo "(large meter minus small meter)" uses "small meter" where the paper otherwise says "mini meter".
- p. 4 Mean difference sentence
(large meter minus small meter)
Judge's note: Page 4 image shows "large meter minus small meter"; the fig 2 axis on the same page and the rest of the text use "mini". The meaning is clear; terminology consistency only.
Second check: none (judge only)
Report a problem with this finding
Typo Comma where a full stop is needed: "The standard error of d is thus 9.4, For the 95% confidence interval".
- p. 5 Precision of estimated limits, first paragraph of page
is thus 9.4, For the 95% confidence
- p. 5 Precision of estimated limits of agreement, para 2
The standard error of d is thus 9.4, For the 95% confidence
Judge's note: Page 5 image shows "is thus 9.4, For the 95% confidence": a comma followed by a capitalised new sentence.
Second check: none (judge only)
Report a problem with this finding
Typo "If we can assume the main difference to be zero" should read "mean difference".
- p. 7 Repeatability, second paragraph
If we can assume the main difference to be zero
- p. 7 Repeatability, second paragraph
If we can assume the main
Judge's note: Page 7 image shows "the main difference to be zero". Two sentences earlier the paper says "The mean difference should here be zero", and the calculation that follows (root mean square of the differences) is the one for an assumed zero mean.
Second check: none (judge only)
Report a problem with this finding
Typo "the same method may be use to analyse the repeatability" should read "may be used"; the acknowledgements also read "We thank of our colleagues", with a word missing or "of" superfluous.
- p. 8 Discussion, first paragraph
the same method may be use to analyse
- p. 9 Acknowledgements
We thank of our colleagues
- p. 8 Discussion
may be use to analyse
- p. 8 Discussion
for this purpose?.
Judge's note: Page 8 image shows "may be use to analyse". "We thank of our colleagues" on page 9 was read from the text layer only; the page 9 image was not opened. The "?." punctuation also listed here is handled under C015.
Second check: none (judge only)
Report a problem with this finding
Typo Doubled punctuation: "universally used for this purpose?."
- p. 8 Discussion, second paragraph
universally used for this purpose?.
- p. 8 Discussion, second paragraph, first sentence
used for this purpose?.
- p. 8
universally used for this purpose?. Two processes
Judge's note: Page 8 image shows the question mark followed by a full stop.
Second check: none (judge only)
Report a problem with this finding
Typo Acknowledgements read "We thank of our colleagues for their interest and assistance": the "of" is stray, or a word such as "many" is missing before it.
- p. 9 Acknowledgements
We thank of our colleagues for their interest
- p. 9 Acknowledgements, first line
We thank of our colleagues
Judge's note: Viewed the full page 9 image. The first line of the acknowledgement paragraph reads exactly "We thank of our colleagues for their interest and assistance, including Dr David Robson...". The sentence is ungrammatical as printed. Which word was intended ("many of", "all of", or no "of") cannot be told from the page, but the defect is plain.
Second check: none (judge only)
Report a problem with this finding
Typo Reference 1 reads "The Statistician 1983; 32, 307-317." with a comma after the volume, where references 2-6 use a colon (e.g. "Arch Dis Child 1978; 53: 509-11."); page ranges are also given in full in references 1 and 5 ("307-317", "302-305") but abbreviated in references 2, 3, 4 and 6.
- p. 9 Reference 1
1983; 32, 307-317.
- p. 9 Reference 1
The Statistician 1983; 32, 307-317.
- p. 9 Reference 2
Arch Dis Child 1978; 53: 509-11.
Judge's note: Viewed the full page 9 image. Reference 1 has "32, 307-317"; references 2 to 6 all have volume followed by a colon ("53: 509-11", "34: 807-08", "i: 567-69", "41: 302-305", "56: 445-9"). The page-range style is mixed as well. This is a formatting inconsistency within the reference list only; nothing a reader would be misled by.
Second check: none (judge only)