LLaMA: Open and Efficient Foundation Language Models
2023 · paper · AI and machine learning
Introduces LLaMA, a set of language models from 7B to 65B parameters trained only on publicly available data. LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with Chinchilla-70B and PaLM-540B.
Checked: PDF from arxiv.org, fetched 4 Oct 2026. Page numbers are those of that PDF.
8 findings
Substance Section 3.1: "LLaMA-65B outperforms Chinchilla-70B on all reported benchmarks but BoolQ", but Table 3 gives BoolQ 85.3 for LLaMA-65B and 83.7 for Chinchilla-70B, so LLaMA-65B is ahead on BoolQ too.
- p. 5 Section 3.1, second paragraph
forms Chinchilla-70B on all reported benchmarks but BoolQ
- p. 4 Table 3, Chinchilla 70B row, BoolQ
83.7
- p. 4 Table 3, LLaMA 65B row, BoolQ
85.3
- p. 5 Section 3.1, second paragraph
LLaMA-65B outperforms Chinchilla-70B on all reported benchmarks but BoolQ
Only the authors can say what is right here.
Judge's note: Table 3 on the page 4 image: Chinchilla 70B has BoolQ 83.7, PIQA 81.8, SIQA 51.3, HellaSwag 80.8, WinoGrande 74.9; LLaMA 65B has 85.3, 82.8, 52.3, 84.2, 77.0. LLaMA-65B is higher in every column Chinchilla reports, including BoolQ. There is only one Chinchilla row, and PaLM-540B is dealt with separately in the next sentence. Either the sentence or a table value is wrong; the authors must say which.
Second check: another model agreed
Report a problem with this finding
Substance Chinchilla-70B overall 5-shot MMLU is "67.5" in Table 9 (and Table 10) but "67.6" in the "All" row of Table 16.
- p. 7 Table 9, Chinchilla 70B, Average
67.5
- p. 18 Table 16, row 'All', Chinchilla column
67.6
- p. 18 Table 16, All / group rows
43.9 60.0 67.6
- p. 7 Table 9, Chinchilla Average
Chinchilla 70B 63.6 54.9 79.3 73.9 67.5
Only the authors can say what is right here.
Judge's note: Read Tables 9 and 10 on the page 7 image and Table 16 on the page 18 image. Chinchilla's four domain values are identical in both tables (63.6, 54.9, 79.3, 73.9); only the overall value differs. The overall values of GPT-3 (43.9), Gopher (60.0) and all four LLaMA models agree between the tables. No statement in the text explains a different averaging. The difference is 0.1 and does not change any comparison. The GPT-3 domain differences some reviewers attached here are judged under C022.
Second check: another model agreed
Report a problem with this finding
Substance GPT-3 175B MMLU domain scores differ between Table 9 (Humanities "40.8", Social Sciences "50.4", Other "48.8") and Table 16 (Humanities "40.6", Social Science "50.5", Others "49.0").
- p. 7 Table 9, GPT-3 175B row (Humanities, STEM, Social Sciences, Other)
40.8
- p. 7 Table 9, GPT-3 175B row, Social Sciences and Other
50.4
- p. 18 Table 16, summary rows, GPT-3 column
40.6
- p. 18 Table 16, Social Science summary row, GPT-3 column
50.5
Only the authors can say what is right here.
Judge's note: Read the GPT-3 row of Table 9 on the page 7 image and the summary rows of Table 16 on the page 18 image. STEM (36.7) and the overall value (43.9) agree; the other three domains differ by 0.1 to 0.2 in both directions, so rounding does not explain it. Gopher, Chinchilla and all LLaMA domain values are identical in the two tables. The text gives no different averaging rule or source for the two tables. Which set is right cannot be told from the PDF.
Second check: another model agreed · checked by hand
Report a problem with this finding
Reference or label Table 10 gives the size of GLM as "120B"; the only GLM model the paper cites is "Glm-130b: An open bilingual pre-trained model" (Zeng et al., 2022).
- p. 7 Table 10, GLM row
120B
- p. 16 Reference list, Zeng et al. 2022
130b: An open bilingual pre-trained model
Judge's note: Table 10 on the page 7 image shows "GLM 120B 44.8". The reference list entry for Zeng et al. (2022) on page 16 is titled GLM-130B, and the text mentions GLM only with that citation (introduction and related work). No 120B GLM model is cited or described, so the size label does not match the paper's own reference. Rated as a label error; the score itself is not in question.
Second check: another model agreed
Report a problem with this finding
Reference or label Table 11 caption: "Scores were obtained using the PerplexityAPI"; the text and footnote 3 name the scorer as PerspectiveAPI (https://perspectiveapi.com/).
- p. 8 Table 11 caption
were obtained using the PerplexityAPI
- p. 8 Section 5.1, first paragraph
evaluated by making a request to PerspectiveAPI
- p. 8 Table 11 caption
Scores were obtained using the PerplexityAPI
- p. 8 Sec 5.1
PerspectiveAPI
Judge's note: Read the caption, Section 5.1 and footnote 3 on the page 8 image. The body text says the toxicity score is "evaluated by making a request to PerspectiveAPI" and refers to "the third-party PerspectiveAPI"; a text search finds "Perplexity" as an API name only in this caption. Wrong name for the tool, not a data error.
Second check: another model agreed
Report a problem with this finding
Reference or label Table 15 labels the model "BLOOM-175B", while the paper's own reference for it is titled "Bloom: A 176b-parameter open-access multilingual language model".
- p. 11 Table 15, second row
BLOOM-175B
- p. 15 Reference list, Scao et al. 2022
Bloom: A 176b-
Judge's note: Confirmed on the page 11 image (row label BLOOM-175B, directly under OPT-175B) and on the reference-list crop (Scao et al. 2022, "Bloom: A 176b-parameter ..."). The inconsistency is internal to the paper: the cited title gives 176B. The label looks carried over from the OPT-175B row above. The row's data are not affected.
Second check: another model agreed
Report a problem with this finding
Wording Section 2.3: "vary the learning rate and batch size with the size of the model (see Table 2 for details)", but Table 2 lists batch size 4M for all four models and the Figure 1 caption says "All models are trained with a batch size of 4M tokens"; only the learning rate varies.
- p. 3 Section 2.3, last sentence
steps, and vary the learning rate and batch size with
- p. 3 Table 2, batch size column
4M
- p. 3 Figure 1 caption
All models are trained with a batch size of 4M tokens.
- p. 3 Figure 1 caption
All models are trained
Only the authors can say what is right here.
Judge's note: Page 3 image: the Table 2 batch size column is 4M in every row (6.7B, 13.0B, 32.5B, 65.2B); the learning rate is 3.0e-4 for the two smaller models and 1.5e-4 for the two larger. The Figure 1 caption confirms 4M tokens for all models. No statement elsewhere in the paper explains a batch size that varies. Either the sentence or the table is inaccurate; only the authors can say which.
Second check: another model agreed
Report a problem with this finding
Wording Section 3.4 and Table 7 caption: "LLaMA-65B outperforms Minerva 62B on GSM8k, although it has not been fine-tuned on mathematical data", but Table 7 shows this only with maj1@k (69.7 vs 68.5); without voting LLaMA-65B is lower (50.9 vs 52.4).
- p. 5 Section 3.4, last sentence
65B outperforms Minerva-62B, although it has not
- p. 6 Table 7 caption
outperforms Minerva 62B on GSM8k, although it has
- p. 6 Table 7, Minerva 62B, GSM8k
52.4
- p. 6 Table 7, LLaMA 65B, GSM8k
50.9
Only the authors can say what is right here.
Judge's note: Table 7 on the page 6 image: Minerva 62B GSM8k 52.4, +maj1@k 68.5; LLaMA 65B GSM8k 50.9, +maj1@k 69.7. The text on page 5 says the comparison is made 'with and without maj1@k' and then states the claim without qualification. The claim is true for one of the two columns only, so it overstates. No explanation elsewhere in the paper.
Second check: another model agreed
Report a problem with this finding
26 minor (typos and the like; most checked by the judge only)
Wording Introduction: "the performance of a 7B model continues to improve even after 1T tokens", but Table 2 and Figure 1 show the 7B model was trained on 1.0T tokens only, so nothing is reported beyond 1T.
- p. 1 Introduction, end of second paragraph (right column)
improve even after 1T tokens
- p. 3 Table 2, 6.7B row, n tokens
1.0T
- p. 3 Figure 1 caption
were trained on 1.0T tokens
Judge's note: Page 1 text, and Table 2 (6.7B row: 1.0T) and Figure 1 (7B curve ends at 1000 billion tokens) on page 3. The intended meaning is almost certainly 'still improving at 1T tokens'; as worded it refers to a range with no data. Loose wording only.
Second check: another model agreed
Report a problem with this finding
Wording Section 3 says LLaMA is compared with "GPT-J (Wang and Komatsuzaki, 2021)", but no GPT-J result appears in any table or in the results text.
- p. 4 Section 3, paragraph before 'We evaluate LLaMA on free-form'
GPT-J (Wang and Komatsuzaki, 2021), and GPT-
Judge's note: Page 4 image for the sentence. A search of the full text finds 'GPT-J' only in this sentence and in the reference list entry (GPT-J-6B). The tables on pages 4-6 have no GPT-J row. The announced comparison is not reported; this is a minor inconsistency.
Second check: another model agreed
Report a problem with this finding
Wording Section 3.2: "On both benchmarks, LLaMA-65B achieve state-of-the-arts performance in the zero-shot and few-shot settings", but in Table 4 the best NaturalQuestions 0-shot score is LLaMA-33B (24.9, bold), above LLaMA-65B (23.8).
- p. 5 Section 3.2
LLaMA-65B achieve state-of-the-arts performance
- p. 4 Table 4, LLaMA 33B, 0-shot
24.9
- p. 4 Table 4, LLaMA 65B, 0-shot
23.8
Judge's note: Table 4 on the page 4 image: 0-shot 33B 24.9 (bold), 65B 23.8; the best external model is PaLM-540B at 21.2. LLaMA-65B still beats all earlier models, so the claim holds against prior work; it is imprecise only in that the paper's own 33B model is higher in that one cell.
Second check: another model agreed
Report a problem with this finding
Wording PaLM 62B five-shot MMLU is "53.7" in Table 9 (Average) but "55.1" in Table 10, with no explanation of the difference.
- p. 7 Table 9, PaLM 62B, Average
53.7
- p. 7 Table 10, PaLM 62B
55.1
- p. 7 Table 9
62B 59.5 41.9 62.7 55.8 53.7
- p. 7 Table 10
PaLM 62B 55.1
Only the authors can say what is right here.
Judge's note: Both values seen on the page 7 image. Chinchilla (67.5) and LLaMA 65B (63.4) agree between the two tables. The text for Table 10 says only that "All the reported numbers are from the corresponding papers", which suggests the two values come from different source papers, but the paper does not say so and presents both as 5-shot MMLU for the same model. Table 10 also prints 55.1 for Flan-T5-XXL two rows below, so a copy slip cannot be excluded. Only the authors can say which applies; rated lower than the reviewers did because a different-source explanation is plausible.
Second check: another model agreed
Report a problem with this finding
Wording Section 5.2: "Our model is particularly biased in the religion category (+10% compared to OPT-175B), followed by age and gender"; in Table 12 LLaMA's highest score is sexual orientation (81.0), and by gap to OPT the order after religion is gender, sexual orientation, age.
- p. 9 Section 5.2, second paragraph
OPT-175B), followed by age and gender
- p. 9 Table 12, LLaMA column
81.0
Judge's note: Read Table 12 and the sentence on the page 9 image. The +10 for religion is right (79.0 against 68.6). Read as gaps to OPT-175B, gender (+4.9) comes before age (+2.3), and sexual orientation (+2.4) sits between them and is not named. Read as absolute scores, sexual orientation (81.0) and physical appearance (77.8) are above age (70.1) and gender (70.6). The sentence is loosely worded rather than plainly wrong: age and gender are among the categories where LLaMA is above OPT.
Second check: another model agreed
Report a problem with this finding
Wording Section 6: "we used 2048 A100-80GB for a period of approximately 5 months ... would have cost around 2,638 MWh under our assumptions"; under the paper's formula (400 W, PUE 1.1) 2048 GPUs for five months give about 3,240 MWh, and 2,638 MWh corresponds to about four months.
- p. 10 Section 6
for a period of approximately 5 months to develop
- p. 10 Section 6
els would have cost around 2,638 MWh under our
Only the authors can say what is right here.
Judge's note: Read Section 6 on the page 10 image and Table 15 with its caption on the page 11 image (400W per GPU, PUE 1.1, 0.385 kg CO2e per kWh). 2048 GPUs for 150 days give about 3,244 MWh; 2,638 MWh works back to about 122 days of all 2048 GPUs. The emission figure follows from the energy (2,638 x 0.385 = 1,015.6, printed 1,015). The paper does not say the GPUs were only partly used, and "approximately" gives some slack, so this is a roughly 20% mismatch between the stated duration and the energy rather than a clear error; the authors would have to say which figure is right.
Second check: another model agreed
Report a problem with this finding
Wording Table 15, BLOOM-175B row: "1,082,880 475 MWh 183"; the stated formula (GPU-h × 400W × PUE 1.1) gives 476.5 MWh for 1,082,880 GPU-hours, not 475 MWh.
- p. 11 Table 15, BLOOM-175B row
475 MWh
- p. 11 Table 15, BLOOM-175B row, GPU-hours
1,082,880
- p. 10 Section 6, formula
Wh = GPU-h×(GPU power consumption)×PUE
- p. 11 Table 15, LLaMA-65B row
1,022,362 449 MWh 173
Only the authors can say what is right here.
Judge's note: Read Table 15 and the formula on the page images. With 400 W and PUE 1.1 (both stated in the caption), every other row reproduces when truncated to a whole number: OPT 356.2 -> 356, LLaMA-7B 36.3 -> 36, 13B 59.5 -> 59, 33B 233.4 -> 233, 65B 449.8 -> 449. So the paper truncates, and LLaMA-65B's 449 is consistent with that convention (not an error). BLOOM gives 476.5, which neither truncates nor rounds to 475; it is the only row that does not follow. The carbon figure 183 fits either value (475 × 0.385 = 182.9; 476.5 × 0.385 = 183.4). The gap is about 1.5 MWh (0.3%), so the effect is small; either the MWh cell or the GPU-hours cell is slightly off, and only the authors can say which. No text explains a different power or PUE for BLOOM.
Second check: another model agreed
Report a problem with this finding
Typo Introduction, first sentence: "Large Languages Models (LLMs)" should read "Large Language Models".
- p. 1 Introduction, first sentence
Large Languages Models (LLMs)
- p. 1
Large Languages Models (LLMs) trained on mas-
Judge's note: Read on the page 1 image; the plural 'Languages' is printed. The title and the rest of the paper use 'Language Models'.
Second check: none (judge only)
Report a problem with this finding
Typo Introduction: "The resulting models, called LLaMA, ranges from 7B to 65B parameters": plural subject with singular verb ('ranges' should be 'range').
- p. 1 Introduction paragraph 3
The resulting models, called LLaMA, ranges from 7B
Judge's note: Read on the page 1 image, right column, third paragraph.
Second check: none (judge only)
Report a problem with this finding
Typo Introduction, last paragraph: "compare with others LLMs" should read "other LLMs".
- p. 1 Introduction last paragraph
compare with others LLMs
Judge's note: Read on the page 1 image, right column, last paragraph.
Second check: none (judge only)
Report a problem with this finding
Typo The model of Black et al. (2022) is called "GPT-NeoX" in the Introduction but "GPT-Neo (Black et al., 2022)" in Section 3 and "[GPTNeo]" in Section 2.2.
- p. 4 Section 3
Neo (Black et al., 2022)
- p. 3 Section 2.2
Rotary Embeddings [GPTNeo]
- p. 1 Introduction
GPT-NeoX (Black et al., 2022)
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that GPT-Neo is a separate, earlier model from GPT-NeoX. What the files show: page 1 prints 'GPT-NeoX (Black et al., 2022)', page 4 prints 'GPT-Neo (Black et al., 2022)', page 3 prints 'Rotary Embeddings [GPTNeo]', and the reference entry is 'Gpt-neox-20b'. The naming is inconsistent within the paper for one citation.
Second check: none (judge only)
Report a problem with this finding
Typo Figure 1 caption: "Training loss over train tokens for the 7B, 13B, 33B, and 65 models." The 'B' is missing from '65B'.
- p. 3 Figure 1 caption
13B, 33B, and 65 models.
- p. 3 Figure 1 caption
13B, 33B, and 65 models
- p. 3 Figure 1 caption
7B, 13B, 33B, and 65 models
- p. 3
Training loss over train tokens for the 7B, 13B, 33B, and 65 models.
Judge's note: Read on the page 3 image; the caption prints '65 models' while the legend reads 'LLaMA 65B'.
Second check: none (judge only)
Report a problem with this finding
Typo Grammar slips: Section 3.2 "LLaMA-65B achieve state-of-the-arts performance" (should be 'achieves state-of-the-art'); Section 2.2 "Here are the main difference with the original architecture, and where we were found the inspiration for this change" (stray 'were', 'difference' for 'differences').
- p. 5 Section 3.2
LLaMA-65B achieve state-of-the-arts performance
- p. 3 Section 2.2
where we were found the inspiration
- p. 3 Section 2.2
where we were found the inspiration for this change
Judge's note: Both read on the page images (page 5 Section 3.2; page 3 Section 2.2).
Second check: none (judge only)
Report a problem with this finding
Typo The x-axis label of Figure 1 and of the three lower panels of Figure 2 reads "Billion of tokens" (should be "Billions of tokens").
- p. 8 Figure 2 x-axes
Billion of tokens
- p. 3 Figure 1 x-axis
Billion of tokens
Judge's note: Seen on the Figure 1 crop (page 3) and under the SIQA, WinoGrande and PIQA panels of Figure 2 on the page 8 image. Grammatical slip in an axis label; the meaning is clear.
Second check: none (judge only)
Report a problem with this finding
Typo Table 7 caption: "Minerva 540B uses k = 64 for MATH and and k = 40 for GSM8k": the word 'and' is duplicated.
- p. 6 Table 7 caption
for MATH and and k = 40 for GSM8k
- p. 6 Table 7 caption
k = 64 for MATH and and k = 40 for GSM8k
- p. 6
Minerva 540B uses k = 64 for MATH and and k = 40 for GSM8k
- p. 6
rva 540B uses k = 64 for MATH and and k = 40 for GSM8k).
Judge's note: Read on the page 6 image; 'and and' is printed on one line of the caption, so it is not a line-break artefact.
Second check: code check
Report a problem with this finding
Typo Table 8 caption misspells the benchmark: "MBBP with 3-shot prompts similar to Austin et al. (2021)"; it is MBPP in the table header and the caption's previous sentence.
- p. 6 Table 8 caption
MBBP with 3-shot prompts
- p. 6
MBBP with 3-shot prompts similar to Austin et al.
Judge's note: Read the Table 8 caption on the page 6 image: the previous sentence and the column header say MBPP, this sentence says MBBP.
Second check: none (judge only)
Report a problem with this finding
Typo Section 3.5: "As show in Table 8, for a similar number of parameters" should read "As shown".
- p. 6 Section 3.5
As show in Table 8
Judge's note: Seen on the page 6 image, left column, second paragraph under the tables.
Second check: none (judge only)
Report a problem with this finding
Typo Section 5.3: the parenthesis opened in "(the different choices corresponding to the grammatical function of the pronoun." is never closed.
- p. 9 Section 5.3, end of second paragraph (right column)
grammatical function of the pronoun.
Judge's note: Read the end of the paragraph at the top of the right column on the page 9 image: the sentence ends with a full stop and no closing bracket, and the next paragraph starts "In Table 13".
Second check: none (judge only)
Report a problem with this finding
Typo Section 5.3: "Theses cases correspond to sentences in which the pronoun does not match" should read "These cases".
- p. 9 Section 5.3, 'gotcha' paragraph
Theses cases correspond to sentences
Judge's note: Seen on the page 9 image, right column, "gotcha" paragraph. The same paragraph also has "clearly showing that it capture societal biases".
Second check: none (judge only)
Report a problem with this finding
Typo Section 6 prints "required 34 days on 992 A100-80B"; the GPU is "A100-80GB" in the next sentence ("2048 A100-80GB") and throughout Table 15.
- p. 10 Section 6
required 34 days on 992 A100-80B
- p. 10 Section 6
992 A100-80B
- p. 10 Section 6
2048 A100-80GB
Judge's note: Confirmed on the page 10 image: "992 A100-80B (see their logs)" followed by "2048 A100-80GB". Table 15 on page 11 prints A100-80GB in every row. A dropped letter.
Second check: none (judge only)
Report a problem with this finding
Typo Table 13 caption: "“their/them/some-one’ pronouns than on “her/her/she” and “his/him/he’," opens two quoted terms with a double quote and closes them with a single quote; the parenthesis "(“her/her/she” and “his/him/he”)" lists two of the three pronoun sets in the table.
- p. 10 Table 13 caption
one’ pronouns than on “her/her/she” and “his/him/he’,
- p. 10 Table 13 caption
(“her/her/she” and “his/him/he”)
- p. 10 Table 13 caption
on “their/them/someone’ pronouns than on “her/her/she” and “his/him/he’
Judge's note: Confirmed on the enlarged crop: "their/them/some-one’" and "his/him/he’" close with a single quote mark while opening with a double one; the other quoted terms close properly. This is in the print, not an extraction artefact. The parenthesis omits their/them/someone, which the table reports; that part is a minor incompleteness.
Second check: none (judge only)
Report a problem with this finding
Typo The unit is written "0.385 kg CO2eq/KWh" in Section 6 and "0.385 kg CO2e per KWh" in the Table 15 caption; "KWh" has a capital K throughout (the SI prefix is lower-case k).
- p. 10 Section 6
0.385 kg CO2eq/KWh
- p. 11 Table 15 caption
0.385 kg CO2e per KWh
Judge's note: Both forms confirmed on the page images (page 10 body text, page 11 caption). "CO2eq" versus "CO2e" is an inconsistency within the paper; "KWh" for kWh is used consistently. The value 0.385 is the same in both places, so nothing numeric is affected. Cosmetic.
Second check: none (judge only)
Report a problem with this finding
Typo The reference list spells the same author two ways: "Thorsten Brants, Phillipp Koehn" (Chelba et al. 2013) and "and Philipp Koehn" (Heafield et al. 2013).
- p. 12 Reference list, Chelba et al. 2013
Thorsten Brants, Phillipp Koehn
- p. 13 Reference list, Heafield et al. 2013
and Philipp Koehn
Judge's note: Both spellings confirmed on the crops of pages 12 and 13. The inconsistency is internal to the paper, so one of the two is wrong regardless; that "Philipp" is the correct one is the reviewer's outside knowledge, which I cannot verify from the files, though it is the form used in the ACL proceedings entry.
Second check: none (judge only)
Report a problem with this finding
Typo Hestness et al. 2017 reference lists "Md Patwary, Mostofa Ali, Yang Yang" as if two authors; the same list gives "Mostofa Patwary" as one author in the Shoeybi et al. and Smith et al. entries.
- p. 13 Reference list, Hestness et al. 2017
Md Patwary, Mostofa Ali, Yang Yang
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: that the cited paper's author is one person, Md. Mostofa Ali Patwary, rather than two authors "Md Patwary" and "Mostofa Ali". The page 13 crop shows the entry as quoted. Inside the paper, the Shoeybi et al. and Smith et al. entries (page 15, read in the text layer) print "Mostofa Patwary", which supports the reading that one name was split, but the true author list of the cited work cannot be checked from the files.
Second check: none (judge only)
Report a problem with this finding
Typo Appendix A: "In all settings, we preprend the string Answer these questions:\n to the list of questions and answers"; "preprend" should be "prepend".
- p. 17 Appendix A, second paragraph
we preprend the string
Judge's note: Confirmed on the page 17 crop: the word is printed "preprend" in running text, not in a code or quoted string.
Second check: none (judge only)
Report a problem with this finding
Typo Table 16 labels the category "Other" in the per-subject rows but "Others" in the summary row; Table 9 uses "Other" (and "Social Sciences" where Table 16 has "Social Science").
- p. 18 Table 16, summary rows
Others
- p. 18 Table 16, subject rows (category column)
Other
Judge's note: Confirmed on the page 18 image: the category column prints "Other" for Business Ethics, Clinical Knowledge and so on, and the summary block prints "Others". Table 9's headers (text layer, page 8) are "Social Sciences" and "Other". The meaning is unambiguous and no number is affected; a labelling inconsistency only. The caption's "per domain" for a table with per-subject rows is loose but not wrong, since the table also gives the domain summaries.
Second check: none (judge only)