Ceph: A Scalable, High-Performance Distributed File System
eScholarship (California Digital Library), 2006 · paper · Computer science
Presents Ceph, a distributed file system that separates data from metadata management using a pseudo-random placement function (CRUSH) and semi-autonomous storage devices. Measurements show strong I/O performance and scalable metadata handling, exceeding 250,000 metadata operations per second.
Checked: PDF from usenix.org, fetched 4 Oct 2026. Page numbers are those of that PDF.
1 finding
Reference or label Section 6.2.3: "Figure 11 plots latency (y) versus per-MDS throughput (x) for a 4-, 16-, and 64-node MDS cluster" — the Figure 11 legend labels the three curves "4 MDSs", "16 MDSs" and "128 MDSs"; no 64-MDS curve is plotted.
- p. 11 Section 6.2.3, paragraph beginning 'Figure 11 plots latency'
for a 4-, 16-, and 64-node MDS cluster under
- p. 11 Figure 11 legend
128 MDSs
- p. 11 Section 6.2.3
a 4-, 16-, and 64-node MDS cluster
- p. 11 Section 6.2.3, text on Figure 11
for a 4-, 16-, and 64-node MDS cluster
Only the authors can say what is right here.
Judge's note: Viewed page 11. The text says 64-node; the legend shows three curves labelled 4, 16 and 128 MDSs. No statement elsewhere reconciles the two. The 128-MDS curve turning up at about 1000-1100 ops/sec fits the text's 128-MDS makedirs figure, which suggests the text is the one in error, but only the authors can say which is right.
Second check: another model agreed
Report a problem with this finding
9 minor (typos and the like; most checked by the judge only)
Wording Section 6.1.2: "1 MB writes (not shown) take 13 ms for one replica, and 2.5 times longer (33 ms) for three" — Figure 7's x-axis extends to 1024 KB and does plot the no-replication sync-write point at about 13.5 ms; only the replicated values are off-scale.
- p. 9 Section 6.1.2
1 MB writes (not shown) take 13 ms for one replica, and 2.5 times longer (33 ms) for three
- p. 9 Figure 7 x-axis and no-replication sync-write curve
1024
Judge's note: Viewed Figure 7 on page 9. The red no-replication sync-write curve ends at 1024 KB at roughly 13.5 ms, while the 2x and 3x sync curves leave the 0-20 ms scale before 1024 KB. So "(not shown)" is accurate for the 33 ms value but not for the 13 ms value. The 13 x 2.5 arithmetic is consistent with 33 ms. This is a minor wording imprecision.
Second check: another model agreed
Report a problem with this finding
Wording Figure 10 caption: "efficiency drops no more than 50% below perfect linear (horizontal) scaling for most workloads" — read from the plot, makedirs, makefiles and openshared each fall by more than half from their small-cluster value by 128 nodes; only the two openssh workloads stay near or within 50%.
- p. 10 Figure 10 caption
efficiency drops no more than 50% below perfect linear (horizontal) scaling for most workloads
- p. 10 Figure 10 plot
makedirs
Only the authors can say what is right here.
Judge's note: Viewed Figure 10. Approximate readings, smallest cluster to 128 nodes: makedirs about 1950 to 800, makefiles about 1650 to 620, openshared about 3100 to 800, openssh+include about 4750 to 2350, openssh+lib about 3500 to 2050. On that baseline only 2 of 5 workloads retain 50% or more. Readings are approximate and the paper does not define the baseline for 'perfect linear scaling', so this is a wording query rather than a firm error.
Second check: another model agreed
Report a problem with this finding
Wording Section 6.2.3: "to about 1000 ops per MDS per second (50% efficiency) with 128 MDSs" — Figure 10 plots the makedirs point at 128 nodes at about 800 ops/sec (about 41% of the roughly 1950 small-cluster value); the curve is at about 1050 at 96-112 nodes.
- p. 11 Section 6.2.3, first paragraph on page
to about 1000 ops per MDS per second (50% efficiency) with 128 MDSs
- p. 10 Figure 10, makedirs curve at 128 nodes
makedirs
Judge's note: Viewed Figure 10. The final makedirs point at 128 nodes sits visibly below the 1000 gridline, coinciding with the openshared point at about 800. 'About 1000' is loose rounding, and the parenthetical 'over 100,000 ops/sec total' is consistent with either value (128 x 800 = 102,400). The reading is approximate, so this is a minor consistency point.
Second check: another model agreed
Report a problem with this finding
Wording Section 6.2.3 checkpoint example: "64 thousand nodes with two processors each writing to separate files ... could finish in two seconds. If each file were only 10 MB ... such a cluster could write 1.25 TB/sec" — 128,000 files of 10 MB is 1.28 TB, which over two seconds is 0.64 TB/sec; 1.25 TB/sec corresponds to about 125,000 file creates per second, a different ops-per-file assumption.
- p. 11 Section 6.2.3, right column, top
64 thousand nodes with two processors each writing to separate files
- p. 11 Section 6.2.3, right column, top
cluster could finish in two seconds
- p. 11 Section 6.2.3, right column, top
such a cluster could write 1.25 TB/sec
Only the authors can say what is right here.
Judge's note: Read the passage on page 11. 64,000 x 2 = 128,000 files, and x 10 MB = 1.28 TB. The 'two seconds' at 256,000 ops/sec implies about 4 metadata ops per file, which also makes 'minutes' at 6,000 ops/sec plausible (about 85 s). The 1.25 TB/sec figure instead implies 125,000 files/sec, i.e. 2 ops per file or the whole checkpoint in about 1 s. The downstream figures follow from 1.25 TB/sec: 25,000 OSDs at 50 MB/sec and about 6.25 PB at 250 GB each. The paper does not state the ops-per-file assumption, so the back-of-envelope numbers do not share one evident basis; this is a query for the authors.
Second check: another model agreed
Report a problem with this finding
Typo Section 3.3: "Applications for which coherent behavior is unnecesssary" — misspelling of "unnecessary" (three s).
- p. 4 Section 3.3, last paragraph
behavior is unnecesssary
- p. 4 Section 3.3, last paragraph
unnecesssary
- p. 4
Applications for which coherent behavior is unnecesssary
Judge's note: The crop of page 4 shows "unnecesssary" with a triple s in the typeset text.
Second check: none (judge only)
Report a problem with this finding
Typo Section 5.1: "the clusters physical or logical composition" — missing possessive apostrophe ("cluster's").
- p. 6 Section 5.1, paragraph beginning 'The cluster map hierarchy'
the clusters physical or logical composition
Judge's note: The crop of page 6 shows "the clusters physical or logical composition" with no apostrophe printed.
Second check: none (judge only)
Report a problem with this finding
Typo Section 6.1.1: "bandwidth for writes sizes larger than 32 KB" — should read "write sizes".
- p. 9 Section 6.1.1, continuation paragraph under Figure 6
bandwidth for writes sizes larger than 32 KB
Judge's note: The page 9 image prints "for writes sizes larger than 32 KB". The same page uses "varying write sizes" elsewhere.
Second check: none (judge only)
Report a problem with this finding
Typo Section 10: "where devices fail-ures, expansion and cluster restructuring are the norm" — should read "device failures".
- p. 13 Section 10 Conclusions, first paragraph
where devices fail-
- p. 13 Section 10 Conclusions, first paragraph
where devices fail- ures, expansion and cluster restructuring are the norm
- p. 13 Conclusions
where devices fail- ures
Judge's note: The crop of page 13 shows "where devices fail-/ures, expansion and cluster restructuring are the norm." This is a grammatical slip in the typeset text.
Second check: none (judge only)
Report a problem with this finding
Typo Reference [35]: "Kybos: self-management for distributed brick-base storage" — title probably should read "brick-based storage".
- p. 14 Reference [35]
for distributed brick-base storage
- p. 14 Ref. [35]
distributed brick-base storage
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: the actual title of IBM Research Report RJ 10356 (believed to be 'brick-based storage') cannot be verified from the files. The page 14 crop does print 'brick-base storage', and reference [15] in the same list uses 'brick-based storage systems', which makes a typo likely.
Second check: none (judge only)