ReAct: Synergizing Reasoning and Acting in Language Models

• p. 2 Figure 1, panel (2b), note between Obs 2 and Act 6

  (Here the agent go to cabinet 1, then cabinet 2, 3, then countertop 1 and 2)

• p. 2 Figure 1, panel (2b)

  Act 6: Go to countertop 3

Judge's note: Read on the page 2 image and a zoomed render of the panel. Act 2 = cabinet 1; the note lists cabinet 2, cabinet 3, countertop 1 and countertop 2 as the omitted steps (4 moves), but only Acts 3-5 (3 slots) are free before Act 6. The step numbering in the abbreviated illustration is off by one; it does not affect any result. The note also has the grammar slip "the agent go".

Second check: another model agreed

• p. 4 Section 3.1, Action Space

  or else suggests top-5 similar entities from the Wikipedia search engine

• p. 16 Appendix C.1, Act prompt, third question, Observation 1

  Similar: ['Adam Clayton Powell III', 'Seventh Avenue (Manhattan)', 'Adam Clayton Powell Jr. State Office Building', 'Isabel Washington Powell', 'Adam Powell', 'Adam Clayton Powell (film)', 'Giancarlo Esposito'].

Judge's note: Page 4 image has "top-5 similar entities". On the page 16 image the Act prompt's Observation 1 for the third question lists seven entities (counted). The few-shot prompts were hand-composed, so the count there need not match the API at inference, but the paper does not say so; a small inconsistency with no effect on results.

Second check: another model agreed

• p. 5 Figure 2, right panel, green CoT-SC curve at 21 trials

  CoT-SC

• p. 5 Table 1, row CoT-SC, Fever column

  60.4

Only the authors can say what is right here.

Judge's note: Zoomed render of Figure 2: the green CoT-SC curve reaches the red dashed ReAct line at about 17 trials and ends on or marginally above it at 21, about 60.9-61.0. At this scale 0.5 points is a fifth of a gridline interval and would be visibly below the red line. The other end-points agree with Table 1 (64.6, 62.0 on Fever; HotpotQA values). A read-off from a plot, so the size of the gap is approximate; the difference is small but it changes whether CoT-SC is below or level with ReAct on Fever. Authors should confirm which value is right.

Second check: another model agreed

• p. 6 Section 3.3, ReAct + CoT-SC perform best for prompting LLMs

  they both significantly and consistently outperform CoT-SC across different number of samples

• p. 5 Figure 2, left panel (HotpotQA EM), blue and green curves at 1 trial

  CoT-SC -> ReAct

Judge's note: Figure 2, left panel: the blue (CoT-SC -> ReAct) and green (CoT-SC) curves both start at about 25 EM at 1 trial and separate from 3 trials on; the orange curve is above throughout. In the Fever panel both combined curves are above CoT-SC at every point. Consistent with the heuristic on page 5 (back off when the majority answer occurs less than n/2 times, which cannot happen at n = 1). A single-point exception at the lowest sample count; the claim holds from 3 samples.

Second check: another model agreed

• p. 22 Table 6, ReAct column, first think

  B078GWRC1J and B078GTKVXY are bright citrus

• p. 22 Table 6, search results

  Ginger Fresh Deodorant by Earth Mama

Judge's note: Read the page 22 image. The listing for B078GTKVXY is titled Ginger Fresh, so the thought misdescribes it as shown. The product page opened for B078GWRC1J lists scent options including bright citrus and ginger fresh, so the two listings may be variants of one product with the same options; the page does not show the options for B078GTKVXY. This is a hand-written prompt exemplar and does not affect any reported result, so it is a minor consistency point.

Second check: another model agreed

• p. 1 Footnote *

  Projet page with code

• p. 10 Reproducibility Statement

  https://anonymous.4open.science/r/ReAct-2268/

• p. 14 Appendix A.1

  The code for these experiments are at https://react-lm.github.io/.

• p. 1 Title-page footnote

  Projet page with code: https://react-lm.github.io/.

Judge's note: Page 1 image shows "Projet page with code" in the footnote. Page 10 image shows the anonymous.4open.science URL in the Reproducibility Statement; page 14 (Appendix A.1) gives react-lm.github.io for the code of the same GPT-3 experiments. The misspelling is an objective typo; the leftover anonymised link is an inconsistency in the camera-ready version that the authors would fix by replacing it with the public URL.

Second check: none (judge only)

• p. 2 Figure 1, panel (1) header

  (1) Hotspot QA

• p. 14 Figure 4, panel (1) header

  (1) Hotspot QA

Judge's note: Page 2 image and a zoomed render of Figure 1 show the header "(1) Hotspot QA"; page 14 image shows the same header in Figure 4. Both captions and the rest of the paper say HotpotQA, so this is a misspelling inside the figure graphic.

Second check: none (judge only)

• p. 3 Introduction

  question answering (HotPotQA, Yang et al., 2018)

• p. 8 Table 3 caption

  AlfWorld task-specific success rates (%)

• p. 8 Table 4 caption

  Score and success rate (SR) on Webshop

• p. 5 Table 1 caption

  PaLM-540B prompting results on HotpotQA and Fever.

Judge's note: Checked on page images 1, 2, 3, 4, 5, 7, 8, 15 and 16: each variant is present as quoted. Purely cosmetic; no ambiguity about which benchmark is meant. One reviewer's "Hot Potato" variant does not appear in the paper.

Second check: none (judge only)

• p. 3 Section 2

  (1d, Thought2, 4)

Judge's note: Page 3 image shows "(1d, Thought2, 4)" set without a space, while the same sentence has "Thought 1" and "Thought 3" with a space.

Second check: none (judge only)

• p. 4 Section 3.2, ReAct Prompting

  Similar to Figure 1(d), each trajectory consists of multiple thought-action-observation steps

Judge's note: Page 4 image has "Figure 1(d)"; the same page uses "Figure 1(1d)" in Section 3. Figure 1 on page 2 labels its panels (1a)-(1d), (2a), (2b). The caption does describe "(d) ReAct" within part (1), so the target is unambiguous; a labelling slip only.

Second check: none (judge only)

• p. 5 Section 3.3, ReAct outperforms Act consistently

  Fine-tuning results 3 also confirm the benefit of reasoning traces

• p. 5 Section 3.3, ReAct outperforms Act consistently

  Fine-tuning results 3 also confirm the benefit of reasoning

• p. 5 Section 3.3, first paragraph

  Fine-tuning results 3 also confirm

Judge's note: Page 5 image: the "3" is set on the baseline at text size, not as a superscript. Footnote 3 on that page is about step counts and its mark is attached to "ReAct performance" earlier. The fine-tuning results are in Figure 3 (page 7), so the word "Figure" (or parentheses) is missing.

Second check: none (judge only)

• p. 5 Section 3.2, Finetuning

  we consider a bootstraping approach

• p. 5 Combining Internal and External Knowledge

  As will be detail in Section 3.3

• p. 5 Finetuning paragraph

  bootstraping approach

• p. 6 ReAct performs best for fine-tuning

  halluincated

Judge's note: Both seen on images: page 5 Finetuning paragraph has "bootstraping"; the page 6 crop has "halluincated". The third item in this cluster, "As will be detail in Section 3.3", is also real but is carried by C015.

Second check: none (judge only)

• p. 5 Section 3.2, Combining Internal and External Knowledge

  As will be detail in Section 3.3

Judge's note: Page 5 image, paragraph "Combining Internal and External Knowledge", has "As will be detail in Section 3.3". Wrong verb form.

Second check: none (judge only)

• p. 6 Section 3.3, ReAct performs best for fine-tuning

  (potentially halluincated) knowledge

Judge's note: Visible on the page 6 image in the fine-tuning paragraph; the misspelling is in the typeset text, not an extraction artefact.

Second check: none (judge only)

• p. 6 Section 3.3, observation B

  leading to more reasoning error rate than CoT. we note that

Judge's note: Confirmed on the page image (observation B): full stop after CoT, then lower-case "we".

Second check: none (judge only)

• p. 6 Section 3.3, observation A

  the problem solving trajectory of ReActis more grounded

• p. 6 observation A

  trajectory of ReActis more grounded

Judge's note: The crop shows the monospaced "ReAct" set directly against "is" with no gap, unlike other occurrences of the method name on the page. Present in the typeset page, not an extraction artefact.

Second check: none (judge only)

• p. 33 Appendix E.1, example heading

  Failure: Search error

• p. 6 Table 2, Failure rows

  Search result error

Judge's note: Viewed both crops. Table 2 lists the failure types Reasoning error, Search result error, Hallucination and Label ambiguity. The p. 33 heading is "Failure: Search error", while the neighbouring headings visible in the crops ("Failure: Hallucination", and Label ambiguity per the text layer) use the Table 2 names. The referent is unambiguous, since there is only one search-related category, so this is a shortened label and a minor inconsistency only.

Second check: none (judge only)

• p. 8 Section 4, Results

  even the worse ReAct trial (48%) beats the best trial of both methods

• p. 8 ALFWorld results paragraph

  even the worse ReAct trial (48%)

• p. 8 Results, ALFWorld

  even the worse ReAct trial (48%) beats

• p. 8 Table 3

  ReAct-IM (avg) 55 59 60 55 23 24 48

Judge's note: Confirmed on page 8; a superlative is needed (worst of six trials, set against "best trial"). The second point raised, that 48% is not in Table 3, is not a defect on its own: Table 3 gives only ReAct avg (57) and best of 6 (71), and a worst trial of 48 is consistent with those and with beating Act 45 and BUTLER 37. It cannot be checked from the paper (no per-trial results in the appendix), and 48 happens to equal the ReAct-IM average.

Second check: none (judge only)

• p. 8 Section 4, WebShop paragraph (continued under Tables 3-4)

  and a imitation + reinforcement learning (IL + RL) method

• p. 8 WebShop paragraph

  and a imitation + reinforcement learning (IL + RL)

Judge's note: Confirmed on the page 8 image, first line of text under Tables 3 and 4.

Second check: none (judge only)

• p. 9 Section 5, Language model for decision making, second paragraph

  LLMS have also been increasingly employed

Judge's note: The crop shows full capitals "LLMS" at the start of the paragraph, with "LLMs" three times later in the same paragraph. Not small caps.

Second check: none (judge only)

• p. 9 Language model for decision making, para 2

  “inner monologue",

Judge's note: The crop shows a curly opening quote and a straight double quote after "monologue"; other quotations on the page close with curly quotes.

Second check: none (judge only)

• p. 9 Related work

  does not truly comprise of inner thoughts

Judge's note: Text confirmed on page 9. A usage point; "comprise of" is widely seen and the meaning is clear, so this is at most an optional copy-edit.

Second check: none (judge only)

• p. 9 Related work

  and also relies on expensive datasets

Judge's note: Confirmed on the page 9 image, Language model for decision making, first paragraph.

Second check: none (judge only)

• p. 9 Language model for decision making

  instead rely on expensive human feedback

• p. 9 same paragraph

  make decision about API calls

Judge's note: Read the full page 9 image. Both phrases are printed as quoted. The subject of "rely" is the singular WebGPT, and "make decision" lacks a plural or article.

Second check: none (judge only)

• p. 9 Language model for reasoning

  Recently, (Madaan & Yazdanbakhsh, 2022) systematically studied

Judge's note: Read the full page 9 image; the sentence is printed as quoted. The same page uses the textual form correctly elsewhere ("generalist agents like Reed et al. (2022)"), so this is a citation-style slip, cosmetic only.

Second check: none (judge only)

• p. 10 References, Ahn et al.

  Do as i can, not as i say

Judge's note: Read the reference list on the page 10 image; the Ahn et al. entry prints "Do as i can, not as i say". The first-person pronoun is lowercased, a BibTeX case-folding artefact. No outside knowledge is needed, because the pronoun "I" is always capitalised in English.

Second check: none (judge only)

• p. 15 B.1

  for 4, 000 steps

• p. 15 B.1

  for 2, 000 steps

Judge's note: Read the page 15 crop. All four numbers show visible space after the comma (a math-mode comma), unlike "3,000" in the main text (text layer, Section 3). It is in the typeset page, not an extraction artefact. Purely typographic.

Second check: none (judge only)

• p. 20 Appendix C.2, Act prompt, first claim

  Action2

• p. 20

  Action2 Search[Nikolaj Coster-Waldau]

Judge's note: Read the page 20 image. The first Act example runs Claim, "Action2", "Observation 1", "Action 2". The other two Act examples start with "Action 1". The label has the wrong index and no space. It may reproduce the prompt as used, but it is a mislabel in the published table.

Second check: none (judge only)

• p. 20 Appendix C.2, Act prompt, third claim, Action 2

  Search[Beautiful (Christina Aguilera song)

Judge's note: Read the page 20 image: the line ends after the closing parenthesis with no "]". Every other action on the page is closed. The text layer for page 21 has the same action with the bracket.

Second check: none (judge only)

• p. 20 Act prompt Observation 1

  Outstanding Supporting Actor in a Drama Series..

• p. 21 ReAct prompt Observation 1

  Outstanding Supporting Actor in a Drama Series..

Judge's note: Read the page 20 image and the page 21 crop. Both show two full stops after "Drama Series". It is probably carried over from the retrieved Wikipedia text or the prompt as used; cosmetic.

Second check: none (judge only)

• p. 22 Table 6, ReAct column, first think action

  deodorant less then 50 dollars. I can check B078GWRC1J first.

• p. 22 Table 6, ReAct column, first think

  deodorant less then 50 dollars

• p. 22 Table 6, ReAct column, think step

  less then 50 dollars

Judge's note: Read the page 22 image. "less then 50 dollars" is printed as quoted, and the think action ends at "first." with no "]", while the second think action on the page is closed. It may be the prompt verbatim, but it is a misspelling in the published table.

Second check: none (judge only)

• p. 24 Table 8 caption

  An ReAct prompt on the ALFWorld clean task.

• p. 25 Table 9 caption

  An ReAct-IM prompt on the ALFWorld clean task.

Judge's note: Read the Table 8 caption crop (page 24) and the Table 9 caption on the page 25 image. Both are printed as quoted. "An" is probably carried over from the Table 7 caption ("An Act prompt"). Cosmetic.

Second check: none (judge only)

• p. 25 Appendix D.1

  CoTand Acton several randomly selected examples

• p. 25 Appendix D.1

  Search results are omitted to space.

• p. 25 D.1

  ReAct, CoTand Acton several randomly selected examples

• p. 24 Table 8 caption

  An ReAct prompt on the ALFWorld clean task.

Judge's note: Read the page 25 image. "CoTand" and "Acton" are run together in the typeset page (no space after the monospace method names), so this is not an extraction artefact. "omitted to space" lacks a word; Appendix E.1 (text layer) has "omitted to reduce space". The "An ReAct" caption point also listed in this cluster is the same defect as C044 and is adjudicated there.

Second check: none (judge only)

• p. 25 D.1

  Search results are omitted to space.

Judge's note: Viewed the crop of p. 25: the sentence is printed exactly as quoted. The parallel sentence in Appendix E.1 (main.txt line 2649) reads "Search results and non-representative steps are omitted to reduce space.", which confirms the intended wording. Missing word, cosmetic.

Second check: none (judge only)

• p. 32 Success: False positive, Observation 1

  The was built in venice.

Judge's note: Viewed the crop of p. 32: the text is printed as quoted. The observation is an abridged search result (the authors inserted "..."), so the fragment may be a deliberately shortened or verbatim environment output rather than a slip in the authors' own prose. It reads as garbled either way, but both forms could be legitimate, so this is an observation and not a correction.

Second check: none (judge only)

• p. 33 Failure: Search error

  What does the goddess associated with the goddess frigg consists of what tales?

• p. 33 Failure: Label ambiguity

  (label: Israel-American)

Only the authors can say what is right here.

Judge's note: Rests on outside knowledge: whether the HotpotQA dataset itself contains this question wording and the label "Israel-American". Viewed both crops: the text is printed as quoted. Both strings are presented as dataset content (a HotpotQA question and its gold label), not as the authors' prose, and the paper elsewhere reproduces dataset items verbatim. If the dataset has these forms, they are faithful quotations and not errors; the files here cannot settle that. "Frigg" is capitalised in the search results and in Thought 2 of the same example. Only the authors can say whether these are transcription slips.

Second check: none (judge only)