Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

• p. 1 Figure 1 caption and Sup.(200-2×) point

  compared to other unsupervised and supervised (Sup.) baselines [8]

• p. 20 Table 9, ResNet-200 2× row, Supervised [8]

  80.1

• p. 1 Figure 1 caption

  compared to other unsupervised and supervised (Sup.) baselines [8].

Only the authors can say what is right here.

Judge's note: Figure 1 (p.1): the Sup., Sup.(2×) and Sup.(4×) points sit at about 76.5, 77.8 and 78.9, which match Supervised [8] in Table 9 (p.20). The Sup.(200-2×) star sits at about 80, and Table 9 gives only 'Supervised (ours)' = 80.1 for ResNet-200 2×, with '-' under [8]. Appendix C.2 (p.19) says the [8] baselines 'are also presented in Figure 1', but the 200-2× point cannot come from [8]. The caption does not say this point is the authors' own.

Second check: another model agreed

• p. 19 Table 8

  AutoAugment [87]

• p. 14 Reference [87]

  Randaugment: Practical automated data augmentation with a reduced search space

• p. 14 Reference [87]

  Randaugment: Practical automated data augmentation

Only the authors can say what is right here.

Judge's note: Rests on outside knowledge: AutoAugment is a separate paper by Cubuk et al. that is not in the reference list. [87] is also cited, correctly, for RandAugment on pages 20-21. The RandAugment paper may itself report the AutoAugment baseline number, so [87] could be the source of the 77.6, but the method label then points to the wrong paper. Left for the authors to settle.

Second check: another model agreed

• p. 3 Related work

  ℓ2 consistency loss between the softmax predictions of the teacher and the student

• p. 9 Relationship with Mean Teacher

  this consistency loss is the ℓ2 distance between the logits from a student network

Only the authors can say what is right here.

Judge's note: Both passages were checked on the images. They describe the same loss from the same reference [20], on different quantities (softmax outputs vs logits). The paper does not reconcile the two. Which description matches [20] rests on outside knowledge (the definition in the Mean Teacher paper), so this is left to the authors.

Second check: another model agreed

• p. 6 Semi-supervised training on ImageNet

  BYOL consistently outperforms previous approaches across a wide range of architectures.

• p. 6 Table 2b, ResNet-50 (4×), Top-5 10%

  92.6

• p. 6 Table 2b, ResNet-50 (4×), Top-5 10%

  92.5

• p. 6 Table 2b, ResNet-50 (4x)

  87.9 92.5

Judge's note: Checked in the Table 2b image (p.6). Only one cell is lower, by 0.1, so the wording slightly overstates the result.

Second check: another model agreed

• p. 10 Conclusion

  bridges most of the remaining gap between self-supervised methods and the supervised learning baseline of [8]

• p. 6 Linear evaluation paragraph / Table 1a

  This tightens the gap with respect to the supervised baseline of [8], 76.5%

Only the authors can say what is right here.

Judge's note: Page 6 text and Table 1a: the prior state of the art is InfoMin at 73.0, BYOL is 74.3 and the supervised baseline of [8] is 76.5. Closing 1.3 of 3.5 points is not 'most'. If the gap is measured from SimCLR (69.3), the closed share is about 69%, so the wording depends on the reference point. Flagged as an overstatement for the authors to settle.

Second check: another model agreed

• p. 7 Transfer to other vision tasks

  We fine-tune on trainval2007 and report results on test2007

• p. 23 Appendix D.5

  a R50-C4 backbone with a frozen representation

Only the authors can say what is right here.

Judge's note: Both passages were checked on the images. Neither says which parts are frozen and which are fine-tuned. They may mean that only the conv5 box head is trained, but as printed the two descriptions of the protocol conflict.

Second check: another model agreed

• p. 7 Depth estimation text

  BYOL is better or on par with other methods for each metric.

• p. 7 Table 4b

  Supervised-IN [83] 81.1 95.3 98.8 0.573 0.127

Judge's note: Checked in the Table 4b image (p.7). The table itself bolds the supervised value as best. The gap of 0.002 could be argued to be 'on par', so this is minor.

Second check: another model agreed

• p. 8 Bootstrapping paragraph

  yield performance above 68.4%

• p. 9 Table 5a, τbase 0.9

  68.4

• p. 9 Table 5a

  Moving average of online 0.9 68.4

Judge's note: Checked in the Table 5a image (p.9). 'At least 68.4%' would match the table. This is a trivial wording point.

Second check: another model agreed

• p. 8 Figure 3a, BYOL point at batch size 128

  Decrease of accuracy from baseline

• p. 26 Table 16, batch 128

  69.6±0.5

Only the authors can say what is right here.

Judge's note: Measured on the crop: the y-ticks are about 82.5 px per unit (0 at y≈313, −3 at y≈561). The BYOL point at 128 is at y≈527, about −2.59; the SimCLR point at 256 is at y≈593, about −3.39. Other points match Table 16 to within about 0.1, and Figure 3b matches Table 17. The figure may be based on different runs or seeds; only the authors can say which values are right.

Second check: another model agreed

• p. 10 Importance of a near-optimal predictor

  (≈ 25% top-1)

• p. 31 Table 22, diagonal λpred = µproj

  20.5±2.0

• p. 10 Section 5, Importance of a near-optimal predictor

  yields poor results (≈ 25% top-1)

• p. 31 Table 22

  25.7±6.6

Only the authors can say what is right here.

Judge's note: Read Table 22 on page 31 from the image. The diagonal maximum is 20.5 (λ=μ=5). '≈25%' may be meant as a loose upper figure for the poor cells, but it does not match the condition the sentence describes. This is a wording and pointer ambiguity for the authors.

Second check: another model agreed

• p. 22 Table 10, Caltech-101 row

  101 3060 2550 510 6084

• p. 22 D.1 Caltech-101 bullet

  5 random splits of 25 training images per category, with 5 validation images per category

Only the authors can say what is right here.

Judge's note: Images of Table 10 and D.1 on page 22 checked: 2550/25, 510/5 and 3060/30 all give 102. The likely reason is that the background class is included, but the paper does not say so, which leaves the class count inconsistent.

Second check: another model agreed

• p. 27 Eq. 10

  ln f(vi, v′i) / (1/B Σj exp f(vi, v′j))

Only the authors can say what is right here.

Judge's note: Page 27 image. Eq. 10 has a bare f in the numerator and exp f in the denominator. Eq. 11 and Eq. 12 use exp f in the numerator, and Eq. 12 yields +(1/B)Σ f(v_i, v′_i), which requires exp in the numerator. As printed, Eq. 10 is not consistent with itself whichever convention for f is taken: either f is a positive critic (no exp needed in the denominator) or f is a log-score (exp needed in the numerator). The derivation of Eq. 8 proceeds from Eq. 11 and Eq. 12, which are consistent, so the stated result stands.

Second check: another model agreed

• p. 31 Table 21, λ=1, 2, 10

  62.8±1.5

• p. 31 Table 22, µproj=1 row

  60.8±2.9

Judge's note: I checked both tables on the page 31 image. Section I.2 describes Table 22 as 'a similar experiment'. The paper does not say whether the two tables come from independent sweeps. The differences fall within the reported standard deviations, so separate runs with seed variance are a plausible explanation. I treat this as a minor consistency point, not an error.

Second check: another model agreed

• p. 1 Abstract

  state-of-the art methods

Judge's note: Checked on the p.1 page image: the abstract reads 'state-of-the art methods', while the introduction on the same page reads 'state-of-the-art contrastive methods'.

Second check: none (judge only)

• p. 4 Section 3.1, end

  is provided in in Appendix J

• p. 14 Reference [74]

  In Computer Cision and Pattern Recognition, 2019.

• p. 5 Section 3.2

  Furthemore, notice

• p. 4

  4] and Haiku [65] is provided in in Appendix J. 3.2 Int

Judge's note: The p.4 image shows 'is provided in in Appendix J.' within a single line, so this is not a line-break artefact. The reference [74] crop shows 'Computer Cision'. The 'Furthemore' typo also listed in one note is handled under C005.

Second check: code check

• p. 5 Section 3.2, last paragraph

  Furthemore, notice that

• p. 5 Section 3.2, last paragraph

  Furthemore, notice that performing a hard-copy

• p. 5

  Furthemore, notice that performing a hard-copy of the online parameters

Judge's note: The p.5 crop clearly shows 'Furthemore'.

Second check: none (judge only)

• p. 7 Table 3, Fine-tuned, Aircraft column

  88.1

• p. 20 Table 9, Supervised (ours) Top-5, ResNet-152 2×

  95.3

Judge's note: Checked in the Table 3 image (p.7) and the Table 9 image (p.20).

Second check: none (judge only)

• p. 9 Section 5, after Eq. 7

  φ(u1) = pθ(zθ(u1))

Judge's note: Page 9 image shows pθ after Eq. 7. The predictor is defined as qθ in Section 3.1, Figure 2, Algorithm 1 and Section 3.3 (text lines 198, 241, 404, 1307). This is a notational slip.

Second check: none (judge only)

• p. 12 Reference [27]

  Generative adversarial nets. In Neural Information Processing Systems, 2014.

• p. 13 Reference [66]

  Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680

Judge's note: Both entries on pages 12 and 13 list the same authors, title and year. This is a duplicated bibliography entry.

Second check: none (judge only)

• p. 14 Reference [74]

  Computer Cision and

• p. 14 Reference [74]

  Computer Cision and Pattern Recognition

• p. 14 Reference [74]

  In Computer Cision and

Judge's note: The page 14 image confirms the misspelling. Other entries spell the venue correctly.

Second check: none (judge only)

• p. 19 Figure 5 caption

  multiple ResNets architecture

Judge's note: The page 19 image confirms the wording. This is a grammatical slip.

Second check: none (judge only)

• p. 21 Figure 6 caption

  baselines are ran with RandAugment

• p. 21 Figure 6 caption

  Our supervised baselines are ran with RandAugment

Judge's note: The page 21 image confirms the wrong participle.

Second check: none (judge only)

• p. 22 D.1 ImageNet bullet

  we took the last 10009 last images

• p. 22 D.1 Caltech-101 bullet

  the original does not dataset specifies any train/test splits

Judge's note: The page 22 image confirms both errors as printed. They are copy-editing errors, not extraction artefacts.

Second check: none (judge only)

• p. 22 D.1

  standard metrics for each datasets

Judge's note: Page 22 image, sentence before the metrics bullet list: 'We use standard metrics for each datasets:'. The noun should be singular after 'each'.

Second check: none (judge only)

• p. 22 D.1 ImageNet bullet

  we took the last 10009 last images

Judge's note: Page 22 image, ImageNet bullet: the word 'last' appears twice. This is a copy-editing slip.

Second check: none (judge only)

• p. 24 Appendix E heading

  Pretraining on Places 365

Judge's note: Page 24 image: the heading has a space ('Places 365'). The body text ('Places365-Standard', 'BYOL on Places365') and Table 13 use 'Places365'. This is only an inconsistency in styling.

Second check: none (judge only)

• p. 28 Appendix F.6

  (pi, ti)i≤B

• p. 28

  (pi, ti)i≤B with B the batch size, BYOL uses the loss function

Judge's note: Page 28 image (crop C040-b): the targets are introduced as t_i, and the loss formula immediately after uses z_i. This is a notation slip with an evident fix.

Second check: none (judge only)

• p. 28 Appendix F.5

  For comparaison here is the original

• p. 28 Appendix F.4

  In Table 5b we only shows results

• p. 28 Appendix F.4 / F.5

  For comparaison here is the original SimCLR loss

• p. 28 F.5

  For comparaison here is

Judge's note: Page 28 image: all four slips are present as quoted. Fixes: 'comparison', 'show', 'showed', 'changes'.

Second check: none (judge only)

• p. 33 Appendix J.2, class MLP

  def __init(self, name):

Judge's note: The crop of page 33 shows 'def __init(self, name):' with no trailing underscores. In Python this method would never run as the constructor. The fix is evident; this is a cosmetic typo in code listed as pseudo-code.

Second check: none (judge only)