Robust Speech Recognition via Large-Scale Weak Supervision

• p. 4 Section 2.4

  between two and three passes over the dataset

• p. 28 Table 17, Updates

  1048576

• p. 28 Table 17, Batch Size

  256

Only the authors can say what is right here.

Judge's note: Page 4 image shows 2^20 updates and batch size 256; Table 17 (page 28 image) shows Updates 1048576 and Batch Size 256. Section 2.1 (page 2) says audio is broken into 30-second segments. 2^20 x 256 x 30 s is about 2.24 million hours, 3.3 times the 681,070 hours in Table 6. The paper does not state the number of training segments; shorter or dropped segments would raise the figure further, while a larger segment count (for example from segments not counted in the 680k hours) could lower it. The stated range is only approximately off, so author query at low severity.

Second check: another model agreed

• p. 5 Table 1, Parameters column

  39M

• p. 11 Figure 8, x-axis tick labels

  38M

• p. 6 Section 3.3

  million parameters and a 6.7 WER on LibriSpeech test-clean

• p. 11 Figure 8 x-axis

  38M 73M 244M 768M 1549M

Judge's note: Table 1 on the page 5 image and the Figure 8 tick labels on the page 11 image read as quoted; Section 3.3 on page 6 says 39 million. Four of five sizes differ by 1M. The likely cause is rounding versus truncation or English-only versus multilingual vocabulary, but the paper states neither. No reader is materially misled.

Second check: another model agreed

• p. 6 Section 3.4

  Whisper performs well on Multilingual LibriSpeech, out-

• p. 7 Section 3.4, continuation

  and Maestro (Chen et al., 2022b) in a zero-shot

• p. 7 Table 3, Maestro row, MLS column

  Maestro

• p. 7 Table 3 caption

  significantly behind both Maestro, XLS-R, and mSLAM

Only the authors can say what is right here.

Judge's note: Pages 6 and 7 images. Table 3: MLS column has XLS-R 10.9, mSLAM-CTC 9.7, Maestro '-', Whisper 7.3. The table the sentence relies on gives no MLS figure for Maestro, so the claim for Maestro is unsupported within the paper; a Maestro MLS number may exist externally. The "both ... three items" wording in the caption is a cosmetic slip.

Second check: another model agreed

• p. 8 Section 3.5, last paragraph

  posedly having 9,000 hours of translation data

• p. 27 Figure 11, Translation panel, Welsh

  8263

• p. 8 Section 3.5 text

  supposedly having 9,000 hours of translation data

• p. 27 Figure 11, translation panel

  Welsh

Judge's note: Read the Welsh bar label in Figure 11 on the page 27 crop: 8263, ranked 4th after Korean (19938), Chinese (11731) and Japanese (8860), which agrees with the text's 'ranking 4th overall'. 8,263 rounds to 8,000, not 9,000. The text figure is a loose approximation about 9% high; it does not affect the argument.

Second check: another model agreed

• p. 27 Figure 11, Dataset Components

  65% English Speech Recognition

• p. 27 Figure 11, Dataset Components

  (438,218 hours)

• p. 11 Section 4.3

  only 65% of compute is spent on this

Judge's note: Viewed the page 27 image: labels are 17% (117,113 hours), 18% (125,739 hours), 65% (438,218 hours). Recomputed shares are 17.2%, 18.5% and 64.3%, so the first two are rounded down and the third is rounded up beyond normal rounding, apparently so that the labels total 100. A one-point discrepancy. Section 4.3's "only 65% of compute is spent on this" is a compute share and need not equal the hours share, so I confirm the figure label only.

Second check: another model agreed

• p. 24 Table 13, Javanese column, Whisper large-v2 row

  nan

Only the authors can say what is right here.

Judge's note: Viewed the page 24 image: in the third block, Javanese column, the large-v2 row reads "nan" while the other five models have values (107.3, 89.5, 88.6, 67.9, 87.0). The text layer also has "nan" and the paper nowhere explains it. Only the authors can supply the value or the reason.

Second check: another model agreed

• p. 3 Section 2.1, last paragraph

  evaluation datasets we thought were at higher risk of overlap, namely TED-LIUM 3

Judge's note: Page 3 image confirms the sentence. Only one dataset follows "namely". Either the plural is loose or a dataset is missing; minor wording point.

Second check: none (judge only)

• p. 5 Section 3.2, last paragraph

  reference transcripts seperating

• p. 5

  reference transcripts seperating contractions from words with whitespace

• p. 5 Section 3.2, second paragraph

  seperating

Judge's note: I checked the crop of page 5 (right column, top). The misspelling 'seperating' is printed there, so it is not an extraction artefact.

Second check: another model agreed

• p. 6 Section 3.3, paragraph beginning 'Our main findings'

  LibriSpeech clean-test WER of 2.5

Judge's note: Page 6 image, left column, paragraph beginning "Our main findings"; the right column of the same page uses "test-clean" twice.

Second check: none (judge only)

• p. 6 Table 2 row label

  Tedlium

Judge's note: Table 2 on page 6 prints 'Tedlium'. The Figure 2 axis on the same page, Section 3.8, Figure 6, Figure 10 and Appendix A all use the hyphenated capitalised form. A naming inconsistency only; the dataset is not in doubt.

Second check: none (judge only)

• p. 11 Section 4.2, last paragraph

  to decided between these explanations

• p. 11

  speech recognition in order to decided between these explanations

• p. 11 Section 4.2, last paragraph

  in order to decided between these explanations

• p. 8 Section 3.5

  rather transcription data

Judge's note: The crop of page 11 (right column) shows 'to decided'. One reviewer also bundled in 'rather transcription data' on p.8, which I judge under C023. The same reviewer flagged 'makes 55.2% less errors' in the Table 2 caption; I checked it on the p.6 crop. That is a less/fewer usage point and I do not count it as a defect. This verdict covers only the 'to decided' typo.

Second check: another model agreed

• p. 7 Table 3 caption

  behind both Maestro, XLS-R, and

Judge's note: Read the Table 3 caption on the page 7 image. 'both' introduces a list of three models. Cosmetic; delete 'both'.

Second check: none (judge only)

• p. 7 Section 3.4, second paragraph

  limited coverage and room to study Whisper models multi-

Judge's note: Read on the page 7 image, right column. The apostrophe is absent in the image, not only in the text layer. Cosmetic.

Second check: none (judge only)

• p. 8 Section 3.5, last sentence

  included as translation training data rather transcription data

• p. 8

  translation training data rather transcription data according to our

Judge's note: The crop of page 8 (right column) shows the sentence printed without 'than'.

Second check: another model agreed

• p. 20 Appendix A.3

  CoVOST 2

• p. 8 Section 3.5

  CoVoST2 (Wang et al., 2020b)

• p. 20 Appendix A.3

  CoVOST 2 (Wang et al., 2020b)

Judge's note: Viewed page 20 (bold "CoVOST 2" with capital O and a space) and page 26 (caption "BLEU scores on CoVoST2"; the small-caps heading D.3.2 "COVOST 2" is a heading style). The text layer confirms "CoVoST2" throughout the body. Internal inconsistency in capitalisation and spacing.

Second check: none (judge only) · checked by hand

• p. 9 Section 3.8

  sets of videos/podcasts that has been used as ASR benchmarks

Judge's note: Read on the page 9 image, right column. The subject is the plural 'sets'. Cosmetic.

Second check: none (judge only)

• p. 12 Section 4.4

  the WER of Whisper models’ significantly more

Judge's note: Read on the page 12 image, left column. The apostrophe is in the published PDF and nothing follows for it to possess. Cosmetic.

Second check: none (judge only)

• p. 14 References, Amodei et al.

  recognition in english and mandarin. arxiv. arXiv preprint

Judge's note: Viewed the crop of the page 14 reference list: the redundant "arxiv." is printed before the italic "arXiv preprint".

Second check: none (judge only)

• p. 15 References, Collobert et al.

  12(ARTICLE):2493–2537, 2011

Judge's note: Viewed the crop of the page 15 reference list: "12(ARTICLE):2493–2537, 2011" is printed as quoted. The placeholder is not an issue number and should be removed.

Second check: none (judge only)

• p. 19 Appendix A.1, AMI-IHM and AMI-SDM1

  following the stage 0 ad 2 of the s5b recipe

• p. 19 Appendix A.1, AMI-IHM and AMI-SDM1

  stage 0 ad 2 of the s5b recipe

Judge's note: The crop of page 19 (AMI-IHM and AMI-SDM1 bullet) shows 'stage 0 ad 2' as printed.

Second check: another model agreed

• p. 19 Appendix A.2, Kincaid46

  ¡Which

• p. 19 Appendix A.2, Kincaid46

  2018¿ by Jason Kincaid

• p. 19

  blog article ¡Which automatic transcription service is the most accurate - 2018¿ by

• p. 19 Appendix A.2, Kincaid46

  ¡Which automatic transcription service is the most accurate - 2018¿

Judge's note: Viewed the page 19 image: the glyphs ¡ and ¿ are printed around the linked title, so this is in the PDF and not an extraction artefact (the usual result of < and > in LaTeX text mode).

Second check: another model agreed

• p. 20

  We collected the X into English data collected using

Judge's note: Viewed the page 20 image: the CoVOST 2 bullet prints the sentence as quoted.

Second check: another model agreed

• p. 21 Appendix C, English rule 4

  Remove whitespace characters that comes before an apostrophe

Judge's note: Viewed the page 21 image: the rule is printed as quoted, with a plural subject and singular verb.

Second check: none (judge only)

• p. 21 Appendix C, non-English standardization steps 4 and 5

  4. make the text lowercase.

Judge's note: Viewed the page 21 image: items 4 and 5 of the second list begin with lower-case "make" and "replace"; every other list item on the page is capitalised. Purely cosmetic.

Second check: none (judge only)