Mamba: Linear-Time Sequence Modeling with Selective State Spaces

• p. 11 Caption under the 'Induction Heads Extrapolation' plot

  Table 2: (Induction Heads.)

• p. 11 Induction heads extrapolation plot caption

  Table 2: (Induction Heads.) Models are trained on sequence length

Judge's note: Page 11 image: the item captioned Table 2 is a plot of accuracy against test sequence length, placed beside Table 1; the caption says "Full numbers in Table 11". The text refers to it consistently as Table 2 (Section 4.1.2 and the appendix), so no pointer is broken. The label looks deliberate for the side-by-side layout, but it is a figure labelled as a table.

Second check: another model agreed

• p. 2 Section 1, Selection Mechanism paragraph

  a key limitation of prior models: the ability to efficiently select data

• p. 2 Introduction, Selection Mechanism paragraph

  we identify a key limitation of prior models: the ability to efficiently select data in an input-dependent manner

• p. 1 Abstract

  a key weakness of such models is their inability to perform content-based reasoning

Judge's note: Page 2 image (Selection Mechanism paragraph) and page 1 image (abstract) read as quoted. Taken literally the sentence says the opposite of what is meant (the lack of that ability). The intended meaning is clear from context, so this is a wording slip, not a factual error.

Second check: another model agreed

• p. 32 E.2.3 task list

  ARC-easy: an easy subset of ARC-challenge

Only the authors can say what is right here.

Judge's note: Rests on outside knowledge: that the ARC dataset (Clark et al. 2018) is split into disjoint Easy and Challenge sets, so ARC-easy is a subset of ARC, not of ARC-challenge. The page 32 image shows the bullet as quoted, with no citation of its own. Nothing in the paper defines the two sets, so this cannot be verified from the files. It is loose wording in a task list and does not affect any result.

Second check: another model agreed · checked by hand

• p. 1 Section 1, first paragraph

  layer (Bahdanau, Cho, and Bengio 2015) The efficacy of self-attention

• p. 1 Introduction

  (Bahdanau, Cho, and Bengio 2015) The efficacy of self-attention

Judge's note: Page 1 image, Introduction first paragraph: the closing parenthesis of the citation is followed directly by "The efficacy" on the same line, with no period.

Second check: none (judge only)

• p. 1 Introduction

  the very properties that makes it effective

Judge's note: Page 1 image, end of the first Introduction paragraph reads "at the expense of the very properties that makes it effective".

Second check: none (judge only)

• p. 1 Introduction

  Sutskever, Vinyals, and Quoc V Le 2014

Judge's note: Page 1 image shows the text as quoted. The same form recurs elsewhere in the text ("Ramachandran, Zoph, and Quoc V Le 2017", "Ha, Dai, and Quoc V. Le 2017"), so it is a systematic bibliography-style artefact for this author rather than a one-off slip. Cosmetic.

Second check: none (judge only)

• p. 2 Introduction, Hardware-aware Algorithm paragraph

  all prior SSMs models must be time- and input-invariant

Judge's note: Page 2 image, Hardware-aware Algorithm paragraph, reads "all prior SSMs models".

Second check: none (judge only)

• p. 2 Section 2

  broadly related to RNNs, and CNNs, and classical state space models

Judge's note: Page 2 image, first sentence of Section 2, reads as quoted. Punctuation only; could be passed as style.

Second check: none (judge only)

• p. 27 Appendix C, first display equation of the proof

  h(t) = −h(t) + x(t)

• p. 3 Equation (1a)

  h′(t) = Ah(t) + Bx(t)

Judge's note: Page 27 image shows no prime or dot on the left-hand h(t); page 3 image shows equation (1a) with the prime. The intended equation is evident from (1a) and the phrase "leaky integrator", and the rest of the proof (ZOH discretization) is unaffected, so this is a dropped symbol rather than a substantive error.

Second check: none (judge only)

• p. 5 Section 3.2

  Algorithms 1 and 2 illustrates the main selection mechanism

Judge's note: Crop of page 5, Section 3.2 second paragraph, reads "Algorithms 1 and 2 illustrates the main selection mechanism that we use."

Second check: none (judge only)

• p. 8 Section 3.5.2, Filtering Context

  improves monotonicly with context length

Judge's note: Crop of page 8, Filtering Context paragraph, reads "their performance in principle improves monotonicly with context length".

Second check: none (judge only)

• p. 9 Initialization

  based on the HIPPO theory

Only the authors can say what is right here.

Judge's note: Rests on outside knowledge: that the cited method (Gu, Dao, et al. 2020) is styled "HiPPO" in its original publication. The page 9 crop shows "HIPPO theory". Within this paper the capitalisation is consistent: the reference-list titles and Appendix text also print "HIPPO" (text layer). So there is no internal inconsistency, only a possible departure from the original styling. Cosmetic.

Second check: none (judge only)

• p. 14 Figure 7 legend

  S4+FFN

• p. 14 Section 4.4.1

  SaShiMi (S4+MLP) baseline

• p. 14 Section 4.4.1

  the SaShiMi (S4+MLP) baseline

Judge's note: Legend on page 14 reads S4+FFN; the text on the same page reads S4+MLP. "FFN" appears nowhere else in the text extract. FFN and MLP are synonyms, so this is a labelling inconsistency only.

Second check: none (judge only)

• p. 14 Figure 7 caption

  prior state-of-the-art (Sashimi)

• p. 14 Section 4.4

  SaShiMi architecture

Judge's note: Caption on page 14 reads Sashimi; Section 4.4, Tables 4 and 5 and the appendix use SaShiMi (text search found only this one variant spelling).

Second check: none (judge only)

• p. 16 Section 4.6.1, sentence before 4.6.2

  such as MLP (a traditional architecture) MHA (a hybrid attention architecture)

Judge's note: Read on the page 16 image; the two items run together with no conjunction.

Second check: none (judge only)

• p. 24 Appendix A, Selection, second paragraph

  clarify the overloaded use of former

Judge's note: Read on the crop of page 24; the article is absent.

Second check: none (judge only)

• p. 27 B.4

  Linear Randomized Attention (Zheng, C. Wang, and L. Kong 2022) generalize RFA from the perspective of importance sampling, and generalize it

Judge's note: Read on the page 27 image. Singular subject with plural verb, and the verb is repeated awkwardly. Cosmetic.

Second check: none (judge only)

• p. 28 Appendix D, first paragraph

  up to 7× times faster than attention

• p. 28 Appendix D, first paragraph

  up to 7× times faster than attention at sequence length 32K

• p. 28 Speed paragraph

  This the case with our scan operation

Judge's note: Read on the page 28 image. The "This the case" wording also mentioned by one reviewer is handled under C029.

Second check: none (judge only)

• p. 28 Appendix D, Speed paragraph

  This the case with our scan operation

Judge's note: Read on the page 28 image, Speed paragraph; "is" is absent.

Second check: none (judge only)

• p. 28 Appendix D intro

  SSMs are no-longer equivalent to convolution

Judge's note: Read on the page 28 image. The whole of "no-longer" sits on one line (before the line break to "equivalent"), so the hyphen is not a line-break hyphen.

Second check: none (judge only)

• p. 29 Appendix D continuation, paragraph under Table 11

  an each MLP layer stores around 20 bytes

• p. 29 same sentence

  ((assuming mixed-precision training in FP16 or BF16))

• p. 29 Appendix D end, memory paragraph

  an each MLP layer stores

Judge's note: Read the paragraph under Table 11 on the page 29 image and crop. Both 'an each MLP layer' and the doubled opening and closing parentheses are printed as quoted. The parenthetical is a single clause, so the doubling is not nesting. The sum 12 + 20 = 32 is correct.

Second check: none (judge only)

• p. 32 Appendix E.2.3, last sentence

  for almost all models for these task

• p. 32 E.2.3

  higher for almost all models for these task

Judge's note: Read on the page 32 image. The sentence refers to two tasks (HellaSwag and ARC-challenge), and the singular 'task' after 'these' is printed as quoted.

Second check: none (judge only)

• p. 33 Appendix E.3.3, first paragraph

  sequence length of 220, and that HyenaDNA

• p. 33 Appendix E.3.3, first paragraph

  sequence length of 220, and that HyenaDNA used much larger batches of 228

Judge's note: The crop shows a 2 with superscript 2 and then a full-size 0 on the baseline, while 2^24, 2^20, 2^10, 2^23 and 2^28 in the same paragraph are set correctly. The intended value is 2^20 (an unbraced exponent in the source). The meaning is recoverable from context.

Second check: none (judge only)

• p. 34 Appendix E.4.1, Dataset

  sequence length to be a a multiple of 8

• p. 34 E.4.1 Dataset

  to be a a multiple of 8

Judge's note: On the page 34 image both 'a's are on the same line ('sequence length to be a a multiple of 8'), so this is not a line-break join artefact.

Second check: none (judge only)

• p. 35 Appendix E.4.2

  automated metrics of generated samples continually improving throughout training

• p. 35 E.4.2

  However, automated metrics of generated samples continually improving throughout training.

Judge's note: Read on the page 35 image. The sentence is printed as quoted and should read 'continually improved' or 'kept improving'.

Second check: none (judge only)

• p. 35 Appendix E.4.1 continuation, first paragraph on page

  as opposed the default Mamba architecture Mamba-S6

Judge's note: Viewed the crop of PDF page 35 (Appendix E.4.1, paragraph after Figure 10). The printed sentence reads "To disambiguate, we call this Mamba-S4 as opposed the default Mamba architecture Mamba-S6." The word "to" is missing after "opposed".

Second check: none (judge only)

• p. 36 Appendix E.5, End-to-end Inference

  measure time time taken to generate 128 tokens

• p. 35 E.4.1 after Figure 10

  as opposed the default Mamba architecture

• p. 36

  , 32, 64, to 128, and measure time time taken to generate 1

Judge's note: Viewed the crop of PDF page 36 (Appendix E.5, End-to-end Inference). The line prints "and measure time time taken to generate 128 tokens", with "time" twice in a row. One reviewer note also mentions the missing "to" in "as opposed the default" on page 35; that separate defect is covered by C046, so this cluster is confirmed for the duplicated word only.

Second check: code check

• p. 36 E.5 Scan Operation

  separately performs FFTs on the inputs and the filters, multiply them in frequency domain

Judge's note: Viewed the crop of PDF page 36 (Appendix E.5, paragraph after the memory table). The sentence reads "which separately performs FFTs on the inputs and the filters, multiply them in frequency domain, then performs an inverse FFT to obtain the result." The subject is singular (the standard implementation) and the first and third verbs are "performs", so "multiply" should be "multiplies". The missing article before "frequency domain" is a style point and is not counted as a defect.

Second check: none (judge only)