A Simple Framework for Contrastive Learning of Visual Representations
2020 · paper · AI and machine learning
Presents SimCLR, a simple contrastive framework for learning visual representations without labels, studying its components. It finds data augmentation composition, a nonlinear projection head, and larger batches matter; a linear classifier reaches 76.5% top-1 on ImageNet.
Checked: PDF from arxiv.org, fetched 4 Oct 2026. Page numbers are those of that PDF.
6 findings
Substance B.8.2 says "The self-supervised ResNet gets 69.3% top-1 accuracy, 6.8% worse than the supervised model in absolute terms", but B.8.1 gives the supervised ResNet-50 baseline as 76.3%, a gap of 7.0 points.
- p. 18 Appendix B.8.2, text after Table B.5
ResNet gets 69.3% top-1 accuracy, 6.8% worse than the supervised model in absolute terms
- p. 17 Appendix B.8.1, Supervised Baselines
supervised ResNet-50 baseline achieves 76.3% top-1 accuracy on ImageNet, vs. 69.3%
- p. 17 Appendix B.8.1, Supervised Baselines
supervised ResNet-50 baseline achieves 76.3% top-1 accuracy on ImageNet, vs. 69.3% for the self-supervised counterpart
- p. 17 B.8.1 Supervised Baselines
supervised ResNet-50 baseline achieves 76.3% top-1 accuracy
Only the authors can say what is right here.
Judge's note: Viewed pages 17, 18 and 14. The stated accuracies give 76.3 − 69.3 = 7.0, not 6.8. Using the Table B.3 value (76.4) gives 7.1, and the 90-epoch value (76.5) gives 7.2. The companion 4× gap is consistent (78.3 − 76.5 = 1.8). No printed supervised value yields 6.8 and nothing in the text explains it. Either 6.8 or 76.3 is off; only the authors can say which. The 76.3 vs 76.4 point is handled separately in C018.
Second check: another model agreed
Report a problem with this finding
Reference or label Section 6: "Table 1 shows more numerical comparisons among different methods." Table 1 is the colour-distortion-strength table (SimCLR vs supervised), not a comparison among methods.
- p. 8 Section 6, Linear evaluation paragraph
Table 1 shows more numerical comparisons among different methods.
- p. 5 Table 1 caption
under varied color distortion strength (see Appendix A) and other data transformations
- p. 8 Section 6, Linear evaluation paragraph
Table 1 shows more numerical compar-
Judge's note: Page 8 image has the sentence; page 5 image shows Table 1 holds only SimCLR and Supervised rows across colour-distortion strengths and AutoAug. Table 6 is already cited at the start of the same paragraph, so the intended target is most likely Figure 1 (the method comparison plot); possibly Table 6. The pointer is wrong either way.
Second check: another model agreed
Report a problem with this finding
Reference or label The Table B.5 caption says "See also Figure 8 for results with the ResNet (4×) architecture", but Figure 8 is the projection-head plot; the ResNet-50 (4×) transfer results are in Table 8.
- p. 18 Table B.5 caption
See also Figure 8 for results with the ResNet (4×) architecture.
- p. 6 Figure 8 caption
Linear evaluation of representations with different projection heads
Judge's note: Viewed the Table B.5 caption on page 18 and Figure 8 on page 6. Figure 8 is "Linear evaluation of representations with different projection heads" and has no transfer-learning content. Table 8 is the transfer-learning comparison, and B.8.2 on page 17 itself points to "Table 8 of the text". The pointer should read Table 8.
Second check: another model agreed
Report a problem with this finding
Reference or label Footnote 10: "Figure B.1 shows using a square root learning rate scaling can improve performance of ones with small batch sizes"; Figure B.1 is top-5 accuracy with linear scaling, and the linear vs square-root comparison is in Table B.1.
- p. 7 Footnote 10 (Figure 9)
Figure B.1 shows using a square root learning rate scaling can improve performance
- p. 13 Appendix B.1, second paragraph
In both Figure 9 and Figure B.1, we use a linear scaling of learning rate
- p. 7 Footnote 10 (to Figure 9)
Figure B.1 shows using a square root learning rate scaling can improve performance of ones with small batch sizes.
- p. 13 Figure B.1 caption
Linear evaluation (top-5) of ResNet-50 trained with
Judge's note: Page 13 image: Appendix B.1 says "In both Figure 9 and Figure B.1, we use a linear scaling of learning rate" and "A comparison is presented in Table B.1". The Table B.1 caption states the conclusion the footnote describes. Figure B.2 uses square-root scaling but has no comparison with linear. The footnote should point to Table B.1.
Second check: another model agreed
Report a problem with this finding
Reference or label Appendix B.3: "Figure B.3 shows the top-1 accuracy." Figure B.3 is the eigenvalue distribution of the projection matrix; the supervised top-1 accuracies are in Table B.3.
- p. 14 Appendix B.3, first paragraph
Figure B.3 shows the top-1 accuracy.
- p. 15 Figure B.3 caption
Squared real eigenvalue distribution of linear projection
- p. 14
Figure B.3 shows the top-1 accuracy. We observe that there is no significant
Judge's note: Page 14 image: the sentence sits directly above Table B.3 (top-1 of supervised models trained longer). Page 15 image: the Figure B.3 caption is "Squared real eigenvalue distribution of linear projection matrix", and B.4 cites it correctly for that. The pointer should be Table B.3.
Second check: another model agreed
Report a problem with this finding
Wording B.8.2 says the supervised ResNet-50 outperforms the self-supervised model on "most (10 of 12) datasets with fine-tuning", but the fine-tuned rows of Table B.5 show supervised numerically higher on 9 of 12.
- p. 17 Appendix B.8.2
and most (10 of 12) datasets with fine-tuning
- p. 18 Table B.5, Fine-tuned rows
88.2 97.7 85.9 75.9 63.5 91.3 88.1
- p. 18 Table B.5, Fine-tuned rows
88.2 97.7 85.9 75.9 63.5 91.3 88.1 84.1 73.2 89.2 92.1 97.0
- p. 17 B.8.2
most (10 of 12) datasets with fine-tuning
Only the authors can say what is right here.
Judge's note: Read the fine-tuned rows of Table B.5 on the page 18 image. Supervised is higher on Food (88.3 vs 88.2), CIFAR100 (86.4 vs 85.9), SUN397 (64.3 vs 63.5), Cars (92.1 vs 91.3), VOC2007 (85.0 vs 84.1), DTD (74.6 vs 73.2), Pets (92.1 vs 89.2), Caltech-101 (93.3 vs 92.1) and Flowers (97.6 vs 97.0), which is 9. SimCLR is higher on CIFAR10 (97.7 vs 97.5), Birdsnap (75.9 vs 75.8) and Aircraft (88.1 vs 86.0). Counting by bold instead: SimCLR is not bold on 6 datasets, and supervised is bold on 11. No reading gives 10. The paper does not define the count, so the authors should say which is intended.
Second check: another model agreed
Report a problem with this finding
9 minor (typos and the like; most checked by the judge only)
Wording Footnote 5: "longer training improves performance of stronger augmentation by ∼ 0.5%"; Table B.3 (ResNet-50) shows +Color 75.6 → 76.5 and +Color+Blur 75.3 → 76.7 from 90 to 500 epochs.
- p. 5 Footnote 5 (Table 1)
longer training improves performance of stronger augmentation by ∼ 0.5%
- p. 14 Table B.3, ResNet-50 rows
75.3
Only the authors can say what is right here.
Judge's note: Footnote 5 and Table B.3 read on the page images. The 90 → 500 epoch gains in Table B.3 are 0.9 and 1.4 points, not about 0.5. The footnote does not say which comparison it means: against the 90-epoch crop-only baseline of 76.5, the best strongly augmented result (76.7) is only 0.2 higher. Only the authors can say which comparison gives ∼0.5%.
Second check: another model agreed
Report a problem with this finding
Wording Appendix A TensorFlow pseudo-code: "def color_drop(x): image = tf.image.rgb_to_grayscale(image); image = tf.tile(image, [1, 1, 3])" uses "image" instead of the parameter x and has no return statement.
- p. 12 Appendix A, TensorFlow pseudo-code for colour distortion
image = tf.image.rgb_to_grayscale(image)
Judge's note: Page 12 image: color_jitter(x) operates on x and returns x, but color_drop(x) ignores x, assigns to image and returns nothing, so random_apply(color_drop, image, p=0.2) would get no output. It is labelled pseudo-code and the intent is clear, hence minor. Fix: use x and add "return x".
Second check: another model agreed
Report a problem with this finding
Typo Section 4.2: "using three different architecture for the head" should read "architectures".
- p. 6 Section 4.2
three different architecture for the head
Judge's note: Visible in the crop of page 6. Singular noun after "three different".
Second check: none (judge only)
Report a problem with this finding
Typo Section 6 (a top-level section) opens with "In this subsection, similar to Kolesnikov et al. (2019)".
- p. 7 Section 6, first sentence
In this subsection, similar to Kolesnikov et al. (2019)
Judge's note: Page 7 image: the heading is "6. Comparison with State-of-the-art", a numbered section, and the first sentence calls it a subsection. Wording slip; the meaning is unaffected.
Second check: none (judge only)
Report a problem with this finding
Typo Table 7 group header reads "Methods using other label-propagation:" while the parallel Table B.4 reads "Methods using label-propagation:".
- p. 7 Table 7, first group header
Methods using other label-propagation:
- p. 15 Table B.4, first group header
Methods using label-propagation:
Judge's note: Both headers read on the page images (pages 7 and 15). In Table 7 no label-propagation group precedes this one, so "other" has no referent; the two parallel tables are worded differently. Cosmetic.
Second check: none (judge only)
Report a problem with this finding
Typo Section 6: "are also significantly better then training from scratch" should read "better than".
- p. 8 Section 6, Semi-supervised learning paragraph
are also significantly better then training from scratch
- p. 8 Semi-supervised learning paragraph
significantly better then training from scratch
Judge's note: Visible on the page 8 image, semi-supervised learning paragraph.
Second check: none (judge only)
Report a problem with this finding
Typo Appendix B.1: "A shown in Figure B.2" should read "As shown in Figure B.2".
- p. 13 Appendix B.1, paragraph after Table B.1
A shown in Figure B.2
Judge's note: Visible on the page 13 image, paragraph after Table B.1.
Second check: none (judge only)
Report a problem with this finding
Typo Appendix B.7 repeats a word: "Figure B.6 shows shows linear evaluation versus fine-tuning for different architectures of choice."
- p. 16 Appendix B.7, sentence introducing Figure B.6
Figure B.6 shows shows linear evaluation versus
- p. 16 B.7
Figure B.6 shows shows linear evaluation
- p. 16
on and fine-tuning. Figure B.6 shows shows linear evaluation v
Judge's note: Viewed the page 16 image. Both instances of "shows" sit on the same line, so this is not a line-break artefact.
Second check: code check
Report a problem with this finding
Typo Appendix C has a misspelling: "The ConvNet is a ResNet that has bewen modified to place significant constraints"; "bewen" should be "been".
- p. 19 Appendix C, DIM/AMDIM bullet
a ResNet that has bewen modified
- p. 19 Appendix C, DIM/AMDIM bullet
The ConvNet is a ResNet that has bewen modified
- p. 19 Appendix C, DIM/AMDIM bullet
ResNet that has bewen modified
Judge's note: Viewed the page 19 image; "bewen" is printed in the DIM/AMDIM bullet.
Second check: none (judge only)