Efficiently Reconstructing Dynamic Scenes One D4RT at a Time
Peer-reviewed conference; CVPR 2026 Best Paper, 2026 · paper · AI and machine learning
Introduces D4RT, a feedforward transformer that infers depth, point correspondences and camera parameters from a single video through a querying mechanism for any point in space and time. The authors report it outperforms previous methods across 4D reconstruction tasks.
Checked: PDF from openaccess.thecvf.com, fetched 5 Oct 2026. Page numbers are those of that PDF.
4 findings
Substance Fig. 3 plots STv2 at a pose accuracy of about 0.86. Under the caption's definition ("1 – error, averaged over ATE / RTE / RPE on Sintel and ScanNet"), Table 6's STv2 row "0.126 0.053 1.052 0.018 0.012 0.324" gives about 0.736.
- p. 4 Figure 3 plot and caption
Accuracy is defined as 1 – error, averaged over ATE / RTE / RPE on Sintel and ScanNet.
- p. 7 Table 6, SpatialTrackerV2 row
0.126 0.053 1.052 0.018 0.012 0.324
- p. 7 Table 6, SpatialTrackerV2 row
SpatialTrackerV2 [59] 0.126 0.053 1.052 0.018 0.012 0.324
- p. 4 Figure 3 caption
ATE / RTE / RPE on Sintel and ScanNet
Only the authors can say what is right here.
Judge's note: Applied the caption's definition to Table 6 (p.7). It reproduces MapAnything's printed 0.475 exactly, and MegaSaM (~0.81), VGGT (~0.83), π3 (~0.885) and D4RT (~0.91) match their plotted positions within reading accuracy. For STv2 the same calculation gives 0.736, below the plot's 0.7–0.9 range shown, yet Fig. 3 (p.4) puts STv2 near 0.86, above MegaSaM and VGGT. Either the figure point or a Table 6 entry (for example Sintel RPE-R 1.052) is inconsistent; only the authors can say which. The RTE naming point in reviewer note 3 is covered under C006.
Second check: another model agreed
Report a problem with this finding
Reference or label Reference [29] reads 'Diederik P Kingma. Adam: A method for stochastic optimization. ICLR, 2015.' and lists a single author.
- p. 10 Reference [29]
Diederik P Kingma. Adam: A method for stochastic opti-mization. ICLR, 2015.
- p. 10 Ref [29]
Diederik P Kingma. Adam: A method for stochastic opti-
Only the authors can say what is right here.
Judge's note: Rests on outside knowledge: the Adam paper (ICLR 2015) is by Diederik P. Kingma and Jimmy Ba. The crop of p.10 confirms that the entry lists only Kingma.
Second check: another model agreed
Report a problem with this finding
Wording "D4RT is 18–300× faster than others" (Table 3 caption) and "our model is 18-300× faster than prior methods" (Sec. 4.2): Table 3's DELTA rows (408 at 10 FPS, 5,770 at 1 FPS vs D4RT 3,890 and 40,180) give only about 9.5× and 7×.
- p. 5 Table 3 caption
D4RT is 18–300× faster than others.
- p. 7 Sec. 4.2, last paragraph
our model is 18-300× faster than prior methods
- p. 7 Sec. 4.2, last paragraph
our model is 18-300× faster than prior methods.
- p. 5 Table 3, DELTA row
408
Only the authors can say what is right here.
Judge's note: Table 3 on p.5 image. Against SpatialTrackerV2 the ratios are 17.5–19×; against DELTA at 24 FPS about 314×. Against DELTA at 10 FPS and 1 FPS they are about 9.5× and 7×, below the stated lower bound. Neither the caption nor the text restricts the range to particular settings or baselines.
Second check: another model agreed
Report a problem with this finding
Wording Sec. 4.4 says 'Unless otherwise noted, we adopt a ViT-L backbone with 6 decoder layers as the default setting'. Tables 7 (✓ row) and 9 (D4RT row) report 0.302/0.257/0.091/0.028/0.245 on Sintel, but the Table 8 ViT-L row reports 0.256/0.214/0.073/0.027/0.191, and no different setting is stated.
- p. 8 Table 8, ViT-L row
0.256 0.214 0.073 0.027 0.191
- p. 8 Table 7, ✓ row / Table 9, D4RT row
0.302 0.257 0.091 0.028 0.245
- p. 8 Sec. 4.4
we adopt a ViT-L backbone with 6 decoder layers as the de-fault setting
- p. 8 Sec. 4.4 intro
Unless otherwise noted, we adopt a ViT-L backbone with 6 decoder layers as the de-fault setting.
Only the authors can say what is right here.
Judge's note: Checked p.8. Table 7 ✓ row and Table 9 D4RT row agree (0.302 0.257 0.091 0.028 0.245). The Table 8 ViT-L row differs on every metric (0.256 0.214 0.073 0.027 0.191). The Table 8 caption and the 'Encoder size' paragraph name no different decoder depth, schedule or data. The appendix is not among the files. Only the authors can say whether Table 8 used a different recipe or whether one set of numbers is wrong.
Second check: another model agreed
Report a problem with this finding
15 minor (typos and the like; most checked by the judge only)
Reference or label Table 1 gives u = "1…h" and v = "1…w" in the Extrinsics/Intrinsics rows, but u = "1…W" and v = "1…H" in the Point Cloud/Depth Map rows, so the coarse-grid axes are swapped.
- p. 3 Table 1, Extrinsics and Intrinsics rows
1...h
- p. 3 Table 1, Extrinsics/Intrinsics rows
1...h 1...w
- p. 3 Table 1, Point Cloud/Depth Map rows
1. ..W 1.. .H
Judge's note: Table 1 on p.3 image: in the Extrinsics and Intrinsics rows, u is 1…h and v is 1…w. In the Point Cloud and Depth rows, u is 1…W and v is 1…H. The caption says only that a coarse (h,w) grid is queried. Nothing in the paper explains why u would pair with h. Probable fix: u = 1…w, v = 1…h.
Second check: another model agreed
Report a problem with this finding
Wording Algorithm 1 line 9 reads "G ← Visible(P)" (comment: "Set visible track pixels as visited"). As printed, this overwrites the occupancy grid instead of adding to it.
- p. 3 Algorithm 1, line 9
G ← Visible(P)
Judge's note: Algorithm 1 on p.3 image. Elsewhere in the algorithm the arrow means assignment (lines 1–3, 7, 8, 10). Line 10 accumulates explicitly with T ← T ∪ P. Line 9 replaces G, which contradicts the comment and the p.4 text ("marks all spatio-temporal pixels it visibly passes through as visited"). The intended update is presumably G ← G ∨ Visible(P).
Second check: another model agreed
Report a problem with this finding
Wording "D4RT achieves state-of-the-art results against competing methods, both with and without known ground-truth intrinsics": in Table 4, without GT intrinsics, SpatialTrackerV2 is better and highlighted on camera-coordinate ADT AJ and APD3D (0.260 / 0.342 vs D4RT 0.240 / 0.319), and CoTracker3 has higher ADT OA (0.937 vs 0.926).
- p. 6 Sec. 4.2
As shown in Tab. 4 (left), D4RT achieves state-
- p. 6 Table 4, w/o GT intrinsics, ADT
0.260 0.342 0.936
- p. 7 Sec. 4.2 continuation
of-the-art results against competing methods, both with and without known ground-truth intrinsics.
- p. 6 Table 4, ADT camera coordinate, w/o GT intrinsics, SpatialTrackerV2
0.260 0.342
Judge's note: Table 4 on the p.6 image: the highlighted best cells in the ADT columns belong to baselines in both settings (OA 0.937 for CoTracker3 also with GT intrinsics). D4RT leads on most other columns, so the unqualified claim and the caption's "superior performance" are overstated rather than wrong.
Second check: another model agreed
Report a problem with this finding
Wording World-coordinate tracking text says "exhibiting strong improvements across metrics", but without GT intrinsics SpatialTrackerV2's ADT APD3D (0.336, highlighted as best) exceeds D4RT's 0.319 in Table 4.
- p. 7 Sec. 4.2, world coordinate paragraph
exhibit-ing strong improvements across metrics
- p. 6 Table 4, w/o GT intrinsics, World ADT APD3D
0.336
- p. 7 Sec. 4.2, world coordinate paragraph
We observe that our model excels at this task as well, exhibiting strong improvements across metrics.
- p. 6 Table 4, ADT, without intrinsics
D4RT (Ours) 0.257 0.345 0.875 0.240 0.319 0.926
Judge's note: Table 4 world-coordinate columns on the p.6 image. D4RT is best on all other world-coordinate cells, including ADT APD3D with GT intrinsics (0.398 vs 0.378). The one exception makes "across metrics" overstated. This is a different claim from C010, which concerns camera-coordinate tracking.
Second check: another model agreed
Report a problem with this finding
Wording "our model strongly outperforms competing baselines on both indoor scenes (ScanNet and Re10K)": on ScanNet in Table 6, π3 has better RPE-R (0.291, highlighted, vs D4RT 0.302), ties on RPE-T (0.010), and trails on ATE only 0.015 vs 0.014.
- p. 7 Sec. 4.3, Camera pose estimation
our model strongly outperforms competing baselines on both indoor scenes (ScanNet and Re10K)
- p. 7 Table 6, π3 row, ScanNet
0.010 0.291
- p. 7 Table 6, ScanNet RPE-R
0.302
- p. 7 Table 6
ε3 [57]
Judge's note: Table 6 on the p.7 image. D4RT clearly leads on Re10K (83.5 vs 78.7), but the ScanNet numbers do not support "strongly outperforms". On Sintel, RPE-R ties MegaSaM at 0.126.
Second check: another model agreed
Report a problem with this finding
Wording Depth-alignment description: "the latter introduces an additional 3D translation term". Scale-and-shift alignment of depth maps for AbsRel uses a scalar depth shift, not a 3D translation.
- p. 7 Sec. 4.3, Depth estimation
the latter introduces an additional 3D translation term
Only the authors can say what is right here.
Judge's note: Depth estimation paragraph on the p.7 image. The paragraph aligns predicted video depth to ground-truth depth and then computes AbsRel. A 3D translation does not fit a per-pixel depth alignment. The authors should say whether they mean a scalar shift.
Second check: another model agreed
Report a problem with this finding
Wording "We evaluate camera pose estimation performance on the same datasets": depth is evaluated on Sintel, ScanNet, KITTI and Bonn, while pose (Table 6) uses Sintel, ScanNet and Re10K.
- p. 7 Sec. 4.3, Camera pose estimation
We evaluate camera pose esti-mation performance on the same datasets.
Judge's note: p.7 image: the Depth estimation paragraph lists Sintel, ScanNet, KITTI and Bonn. Table 6 and its caption list Sintel, ScanNet and Re10K. The datasets overlap only partly.
Second check: another model agreed
Report a problem with this finding
Wording The Table 8 caption says 'Performance clearly improves as the backbone increases from ViT-B to ViT-g', but ATE is 0.073 (L), 0.070 (H), 0.078 (g), and RPE-T is 0.027 (L), 0.028 (H).
- p. 8 Table 8 caption
Performance clearly improves as the backbone increases from ViT-B to ViT-g.
- p. 8 Table 8, ATE column
0.070
- p. 8 Table 8
ViT-H 0.226 0.173 0.070 0.028 0.186
Judge's note: Checked Table 8 on p.8. From ViT-B to ViT-g every metric improves end to end. The trend is not monotonic, though: ViT-g has a worse ATE than ViT-H and ViT-L, and ViT-H has a worse RPE-T than ViT-L. The body text is more careful ('particularly in depth estimation and RPE-R'). The caption's unqualified wording is a mild overstatement.
Second check: another model agreed
Report a problem with this finding
Typo Symbol N is used for the number of scene tokens ("F = E(V) ∈ R^{N×C}", Sec. 2.1) and again for the number of training queries ("over a batch of N sampled queries", Sec. 2.3).
- p. 2 Sec. 2.1
F = E(V ) ∈ RN×C
- p. 3 Sec. 2.3
over a batch of N sampled queries
Judge's note: Checked p.2, Sec. 2.1 (F = E(V) ∈ R^{N×C}) and p.3, Sec. 2.3 ("a batch of N sampled queries"). The two are different quantities. Context keeps them apart, so this is a minor notation clash.
Second check: none (judge only)
Report a problem with this finding
Typo The Fig. 3 caption names the metrics "ATE / RTE / RPE", while Table 6 and Sec. 4.3 use ATE, RPE-T (RPE trans) and RPE-R (RPE rot). "RTE" is never defined.
- p. 4 Figure 3 caption
ATE / RTE / RPE on Sintel and ScanNet
Judge's note: Fig. 3 caption on p.4 vs Table 6 headers and the Camera pose estimation paragraph on p.7. The labels are inconsistent; the intended metrics can be inferred.
Second check: none (judge only)
Report a problem with this finding
Typo Sec. 3.2 calls the method "St4RTracker [13]" twice, while Tables 2 and 4 and reference [13] ("St4rtrack: Simultaneous 4d reconstruction and tracking in the world") use St4RTrack.
- p. 5 Sec. 3.2
St4RTracker [13]
- p. 5 Sec. 3.2, second paragraph
DPM [47] and St4RTracker [13]
- p. 5 Sec. 3.2, second paragraph
St4RTracker [13] and DPM [47] follow
- p. 6 Table 4
St4RTrack [13]
Judge's note: The p.5 image shows "St4RTracker" twice in Sec. 3.2. Table 2 (p.4), Table 4 (p.6) and reference entry [13] use St4RTrack.
Second check: none (judge only)
Report a problem with this finding
Typo Table 3 caption: "number of full-video 3D point tracks that different model can produce" should read "different models".
- p. 5 Table 3 caption
number of full-video 3D point tracks that different model can pro-duce
- p. 5 Table 3 caption
that different model can pro- duce
Judge's note: Checked the Table 3 caption on the p.5 image. Number agreement error.
Second check: none (judge only)
Report a problem with this finding
Typo 'As detailed in Sec. D if the appendix' should read 'in the appendix'.
- p. 8 Sec. 4.4, Training data
As de-tailed in Sec. D if the appendix
- p. 8 text near line 1169
tailed in Sec. D if the appendix
Judge's note: Confirmed on the p.8 image, in the 'Training data' paragraph, right column.
Second check: none (judge only)
Report a problem with this finding
Typo The Table 9 caption reads 'A slight trade-off between depth and camera estimation pose is observed'; it should read 'camera pose estimation'.
- p. 8 Table 9 caption
A slight trade-off between depth and camera estimation pose is observed
- p. 8 Table 9 caption
depth and camera estimation pose is observed
Judge's note: Confirmed on the p.8 image in the Table 9 caption. The words are transposed.
Second check: none (judge only)
Report a problem with this finding
Typo ScanNet appears twice in the references: [6] 'Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 5' and [7] 'ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In CVPR, 2017. 7'.
- p. 9 References [6] and [7]
Scannet: Richly-annotated 3d reconstructions of indoor scenes
- p. 9 References [6] and [7]
Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 5
- p. 9 Reference [7]
ScanNet: Richly-annotated 3D reconstructions of indoor scenes. In CVPR, 2017. 7
Judge's note: Checked p.9. [6] and [7] have the same authors, title, venue and year and differ only in capitalisation. [6] is cited on p.5 and [7] on p.7. This is a duplicate bibliography entry.
Second check: none (judge only)