Spark: Cluster Computing with Working Sets
2010 · paper · Computer science
Proposes Spark, a cluster computing framework for applications that reuse data across parallel operations, such as iterative machine learning and interactive analysis. It introduces resilient distributed datasets and outperforms Hadoop by 10x on iterative jobs.
Checked: PDF from usenix.org, fetched 4 Oct 2026. Page numbers are those of that PDF.
0 findings
None that passed a second check.
5 minor (typos and the like; most checked by the judge only)
Wording ALS listing computes M over "0 until m" (movies) with ".map(j => updateUser(j, Rb, U))", the same function used for the user step, while the text says step 3 is "Optimize M given U".
- p. 4 Section 3.3, ALS code listing
.map(j => updateUser(j, Rb, U))
Only the authors can say what is right here.
Judge's note: The p4 image shows both loops calling updateUser, once with M and once with U. A generic function is possible, but the name points to a copy slip (e.g. updateMovie was meant). Only the authors can say which was intended.
Second check: another model agreed
Report a problem with this finding
Typo Prose says "an accumulator variable called gradient" and "the loop adds to gradient" (code font), but the listing names it grad ("val grad = spark.accumulator(new Vector(D))", "grad += s * p.x", "w -= grad.value").
- p. 3 Section 3.2, logistic regression code
val grad = spark.accumulator(new Vector(D))
- p. 3 Section 3.2, paragraph after the code
variable called gradient (with a value of type V ector)
- p. 3 Section 3.2 text
an accumulator variable called gradient
- p. 3 Section 3.2 prose
we use an accumulator variable called gradient
Judge's note: The p3 image shows the code using 'grad' throughout. The prose sets 'gradient' in typewriter font as the variable's name, and no identifier called 'gradient' appears in the listing.
Second check: none (judge only)
Report a problem with this finding
Typo Missing subject: "For example, if wanted to reuse the errs dataset" should read "if we wanted to reuse".
- p. 3 Section 3.1, paragraph before cachedErrs code
For example, if wanted to reuse the errs
Judge's note: The p3 image confirms the word 'we' is missing in print.
Second check: none (judge only)
Report a problem with this finding
Typo Misplaced comma: "iterations take only 6s, each because they reuse cached data" should read "6s each, because".
- p. 5 Section 5, Logistic Regression paragraph
iterations take only 6s, each because they reuse cached
Judge's note: The p5 image confirms the punctuation as printed.
Second check: none (judge only)
Report a problem with this finding
Typo Typo: "scientific computing an database fields" should read "and database fields".
- p. 6 Section 6, Lineage paragraph
tific computing an database fields
- p. 6 Related Work, Lineage
scien- tific computing an database fields
Judge's note: The p6 image of the Lineage paragraph confirms 'an' is printed.
Second check: none (judge only)