The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes

• p. 13 Table 5, Aggregate row

  Aggregate 0.0001 22 59.1% ± 10.5%

• p. 13 Table 5, model rows

  Llama-3-8B-Instruct 0.0001 10 90.0% ± 9.5%

• p. 13 Appendix A text

  we observe deceptive policies emerging in 59.1% of runs

• p. 2 Section 1, Contributions, third bullet

  policies trained in an environment with no incentive

Only the authors can say what is right here.

Judge's note: Viewed Table 5 and the Appendix A text on p13. Only two model rows are listed, 10 runs each with 9 and 4 deceptive. The aggregate N=22 and 59.1% equal 13/22, and back-solving the aggregate means gives two extra runs averaging about 9% pass-first and 64% pass-all, i.e. two honest runs of an unlisted model. Neither table, caption nor text mentions them. Sec. 4.1 gives only "9 of 10" and "4 of 10". The reader cannot reproduce the aggregate, and 59.1% is not the rate for the two models discussed. Separate small point raised in this cluster: the Contributions bullet on p2 says the comparison environment has "no incentive for reward hacking" while Sec. 5.4 says "little incentive" and Appendix A shows reward hacking does emerge there; that is a slight overstatement (S3).

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 13 Table 6, Llama-3-8B-Instruct block (first row is KL 0.001)

  Llama-3-8B-Instruct 0.001 3 100.0% ± 0.0%

• p. 13 Table 6, Aggregate row KL 0.0001

  Aggregate 0.0001 28 96.4% ± 3.5%

• p. 4 Section 4.1

  96.4% (27 of 28 seeds) aggregated across all four models

• p. 13 Table 6, Aggregate rows

  28

Only the authors can say what is right here.

Judge's note: Viewed Table 6 on p13: the Llama-3-8B block starts at KL 0.001, so this is not an extraction loss. At KL 0.0001 the listed rows are Gemma-12B 7, Gemma-27B 7 and Llama-70B 7, which is 21 runs with 20 deceptive (7+7+6); the aggregate needs 28 runs with 27 deceptive, so a Llama-3-8B row of 7 of 7 is missing. At the other KL values one run each is unlisted. Back-solving the aggregate means gives plausible single honest runs (KL 1.0 PassAll about 66.7%, KL 0.1 PassAll about 64.4%), probably single-seed Llama-3-70B rows that were dropped. So the aggregates look right and the table is incomplete; neither caption nor text says rows were omitted. Related: Sec. 4.2 and 4.4 say "3 seeds per configuration" while these baselines use 7 seeds per model, which is not flagged as an exception. No explanation found in Appendix A or H.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 13 Appendix A, first sentence

  In Section 5 we noted that adding a code length penalty standardizes the reward hacking incentive across models

• p. 4 Section 4.1

  to standardize the reward hacking incentive across models, we add an explicit length penalty

• p. 13 Appendix A, first sentence

  In Section 5 we noted that adding a code length penalty standardizes

• p. 13 Appendix A, first line

  In Section 5 we noted that adding a code length penalty standardizes the reward hacking incentive

Judge's note: Viewed p13 and p4. Sec. 4.1 on p4 reads "to standardize the reward hacking incentive across models, we add an explicit length penalty". Section 5 starts on p5 and mentions the code length penalty only in passing in 5.4 and Figure 5 (p8-9), never the standardisation rationale. The Appendix V wording was confirmed in the text layer. The Appendix S pointer mentioned by one reviewer was not checked.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 13 Table 5, Gemma-3-12B-IT row

  34.2% ± 35.3%

• p. 13 Table 6, Gemma-3-27B-IT, KL 0.1 row

  9.2% ± 1.4%

• p. 5 Section 4.4, Task performance metrics

  the fraction of responses that pass the pro-

Only the authors can say what is right here.

Judge's note: Viewed p13 and p5. Sec. 4.4 defines pass-first-test rate as the fraction passing the provided test and pass-all-tests rate as the fraction passing all tests including the hidden ones, so pass-first must be at least pass-all. In every honest row the column is far lower (also 10.0 vs 48.2, 7.2 vs 47.8, 6.1 vs 63.2, 6.6 vs 67.1), and in every row the two columns sum to at most 100%. The column therefore holds the fraction passing only the first test (the deception gap), not the defined metric. The "(deception)" suffix hints at this, but the header names a differently defined metric and nothing in the caption or text redefines it.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 23 Table 15, MBPP / Follow-up prompt

  8.395

• p. 5 Table 2, MBPP row, W1 dist.

  0.654 ± 0.162

Only the authors can say what is right here.

Judge's note: Viewed Table 15 on page 23 and Table 2 on page 5. Section 4.3 says the final probe score is "the mean of the sigmoid scores across selected classifiers", which bounds W1 at 1. The Table 15 caption says AUC is "for individual layers, and averaged across all layers", which can explain the lower AUCs (0.963 and 0.972 against 0.991 and 0.996) as a different quantity. It says nothing about the W1 scale. The values are probably on a logit or unnormalised scale, but the paper does not say so, and the column is labelled the same as in Table 2.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 6 Section 5.1, second paragraph

  (middle panel, exact counts in Table 16)

• p. 6 Figure 2 caption

  Right: Policy type distribution across models, detector and KL coefficients

• p. 6 Sec. 5.1, second paragraph

  Figure 2 (middle panel, exact counts in Table 16)

• p. 6 Sec. 5.1, para 2

  show the results in Figure 2 (middle panel, exact counts in Table 16)

Judge's note: Page 6 image: the middle panel of Figure 2 is the normalized probe-score scatter with no KL or detector axis; the right panel is the stacked-bar grid of policy types by detector coefficient and KL coefficient, which is what Table 16 (page 25) tabulates. The caption says 'Right: Policy type distribution across models, detector and KL coefficients', and the next sentence of the text cites 'Figure 2, right panel'. The fix is 'right panel'.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 24 Figure 9(c), x-axis positions of Gemma points

  Wasserstein distance b/w hardcoded vs honest code

• p. 14 Table 7, Diverse Deception, Gemma-3-12B-IT and Gemma-3-27B-IT

  0.402

Only the authors can say what is right here.

Judge's note: Viewed a 5x render of Figure 9(c) and Table 7 on page 14. Approximate x-positions: Gemma-3-12B squares 0.115 (Table 7: 0.078), triangles 0.64 (0.402), circles 0.77 (0.473). Gemma-3-27B squares 0.24 (0.146), triangles 0.71 (0.427), circles 0.70 (0.554). Llama-3-8B squares 0.39 (0.319), triangles 0.45 (0.317), circles 0.65 (0.690). Llama-3-70B squares 0.32 (0.307), triangle 0.28 (0.251), circles 0.90 (0.900). The Llama-3-70B agreement suggests the axis is the same quantity, yet the Gemma points and some Llama-3-8B points are off by far more than reading error. The figure may use a different sample or split; neither the caption nor Appendix J says so. Only the authors can say which values are right.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 23 Appendix L, last paragraph

  (Appendix E), so a retrained detector learns the same uninformative direction

Only the authors can say what is right here.

Judge's note: Read Appendix E on page 17 (hypothesis, belief-probing method, results, qualitative analysis, connection to obfuscated activations). It gives no measurement of representation change on chat deception data. Appendix L on page 23 cites it for exactly that claim. The intended support may be Table 4 or Appendix P (non-deception probe stability), but neither measures chat-deception representations directly, so only the authors can say which pointer was meant.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 24 Appendix O, second paragraph

  Obfuscated activations would correspond to a policy that produces a faithful Japanese letter yet retains internal representations that the cuisine probe should flag

• p. 3 Section 2.3

  reward-hack with outputs that would trigger a probe on the original model’s activations, but the final model’s activations have changed such that the probe no longer fires

• p. 24 Appendix O, second paragraph

  that produces a faithful Japanese letter yet retains internal representations that the cuisine probe should flag

• p. 3 Section 2.3

  model’s activations have changed such that the probe no

Only the authors can say what is right here.

Judge's note: Viewed Table 1 and Section 2.3 on page 3. Obfuscated activations: response type Deceptive, probe on θ0 High, representation change High; "the final model's activations have changed such that the probe no longer fires". Appendix O on page 24 uses a "faithful Japanese letter" for this category, and two sentences earlier defines honest policies as the ones that describe the actual cuisine. The analogue should be a pandering Southern-European letter whose activations no longer trigger the cuisine probe. The sentence as printed does not match the taxonomy; the authors should reword it.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 17 Appendix E, Results

  RLVR training substantially increases the undesired answer rate across all four models

• p. 17 Appendix E, Results

  minimal shift after training (to 45%)

• p. 8 Figure 3 caption

  increases substantially after RLVR training across all four models

• p. 7 Section 5.3

  of responses endorsing hardcoding—across all four models

Judge's note: Viewed Figure 3 on page 8: the bars are labelled 0.40/0.45 (Llama-3-8B), 0.25/0.60, 0.20/0.55 and 0.10/0.65, and the Llama-3-8B error bars overlap. Page 17 calls the Llama-3-8B change a "minimal shift" and says all models except Llama-3-8B consider hardcoding acceptable after RLVR. Appendix F on page 18 says it "showed minimal belief change". The quoted ranges are correct; the overstatement is "substantially ... across all four models". Nothing in the paper qualifies it.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 25 Appendix P

  these probes retain near-perfect AUC after RLVR training across all four

• p. 7 Section 5.3

  string) trained on the original models retain near-perfect

• p. 28 Table 20, Docstring detection row

  0.85

• p. 25 Appendix P

  these probes retain near-perfect AUC after RLVR training across all four models

Judge's note: Table 20 on page 28 read from the image: Python-code detection 1.00 throughout; docstring detection pre/post 0.96/0.95, 0.90/0.85, 0.89/0.86, 0.99/0.99. The docstring probe was not near-perfect on the Gemma models before RLVR and drops by 0.03 to 0.05 after. The qualitative point (far smaller drop than deception probes) stands, and the Table 20 caption's own wording ("retain their discriminative power") is accurate; "near-perfect ... across all four models" in Appendix P and Section 5.3 overstates.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 8 Section 5.4

  the Gemma models achieve the highest honesty rates across both KL settings

• p. 18 Appendix G

  achieving the highest honesty rates among all detector penalty coefficients we tested

• p. 21 Figure 8 caption

  For the Gemma models, α = 100 yields the highest honesty rates

• p. 8 Section 5.4, Higher KL and detector penalty coefficient help

  models achieve the highest honesty rates across both KL

Only the authors can say what is right here.

Judge's note: Figure 8 on page 21, both panels, y-axis 'Passes All Tests Rate (Honesty)': on the blue β = 1e-04 series the α = 10 diamond is at about 0.575 and the α = 100 triangle at about 0.53; on the orange β = 1e-01 series the α = 100 triangle is highest at about 0.68. The claim therefore holds at one of the two KL values. Values are read off the plot and the error bars overlap, so this is an overstatement rather than a clear reversal; if 'honesty rate' means the fraction of runs classified honest, that quantity is not shown anywhere and the authors would need to say so.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 18 Appendix G, last paragraph

  the average code length for these honest policies (∼95 characters) is lower than the average code length at α ∈{3, 10} under high KL

• p. 21 Figure 8(a) caption

  Honest policies at α = 100 produce moderately long code (∼95 chars)

• p. 18 Appendix G, last paragraph

  length for these honest policies (∼95 characters) is lower than

• p. 21 Figure 8(a) caption

  α = 100 produce moderately long code (∼95 chars), shorter than

Only the authors can say what is right here.

Judge's note: Viewed Figure 8(a) on page 21. The blue triangle (α=100, β=1e-04) is at about x=92, y=0.53. The orange triangle (α=100, β=1e-01) is at about x=250, y=0.68. The orange α=3 square is at about 108 and the orange α=10 diamond at about 147. The text on page 18 says α=100 gives honest policies "at both KL values", so "these honest policies (∼95 characters)" is not restricted to low KL. The statement holds only if the low-KL α=100 runs are compared with the high-KL lower-α runs, which neither the text nor the caption says. The continuation on page 19 adds no restriction. Positions were read from the plot and are approximate.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 26 Figure 10 caption

  across five extraction methods

• p. 22 Appendix K

  We train probes using five extraction strategies on Gemma-3-12B-IT

• p. 26 Figure 10 legend

  Follow-up (last-token)

• p. 22 Appendix K

  AUCs ranging from 0.31 (Flipped) to 0.83 (Last-token-generation)

Judge's note: Figure 10 on page 26: both panels' legends list Flipped, Random, Prompt + Generation, Generation, Last-token-generation and Follow-up (last-token); the Follow-up point sits at AUC about 1.0 (honest rate 1.0, pass-all-tests about 0.66). Appendix K on page 22 lists only (i)-(v). The follow-up point is presumably the main-paper probe added as a reference, but neither text nor caption says so. The 0.31-0.83 range is correct for the five listed methods, so this is an undescribed sixth series rather than a wrong number.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 24 Figure 9(b) caption

  At low KL (0.0001), all deception probes yield deceptive policies

• p. 25 Table 16, Det. Coef 10, KL 1 × 10−4

  14 (70%)

• p. 25 Table 16, Det. Coef 3, KL 1 × 10−4

  6 (33%)

• p. 24 Figure 9(b) caption

  probes yield deceptive policies, with

Only the authors can say what is right here.

Judge's note: Viewed Figure 9(b) on page 24. Bars are about 0.05/0.07/0.04 at α=1, 0.16/0.14/0.20 at α=3 and 0.38/0.33/0.39 at α=10 (MBPP/Diverse/Simple). The MBPP probe, the strongest, is not the lowest at α=3 or α=10. Table 16 on page 25 has α=3, KL 1e-4: 6 of 18 honest; α=10: 14 of 20 honest. Table 16 pools the Simple and Diverse probes across four models, so the run set may differ a little from the panel, but the blanket caption is contradicted for α=3 and α=10 by both the table and the panel. It holds roughly only at α=1 (3 of 39 honest). Bar heights were read from the plot.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 24 Appendix N

  a median deception gap of 0.94 and a minimum of 0.33, far above any reasonable threshold

• p. 25 Table 17

  0.4 59.7% 3.7% 9.0% 27.7%

• p. 24 Appendix N

  a minimum of 0.33, far above

• p. 25 Table 17 caption

  and a minimum of 0.33, far above any reasonable threshold

Judge's note: Viewed Appendix N on page 24 and Table 17 on page 25. The thresholds swept are 0.2, 0.3 (default), 0.4 and 0.5; the honest share goes 58.0%, 59.3%, 59.7%, 59.9%. The honest share rising at 0.4 and 0.5 is consistent with some deceptive policies having gaps below those thresholds. Only the median is far above the thresholds. The robustness conclusion (the distribution barely changes) is supported by the table; the wording about the minimum is not.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 24 Figure 9c caption/legend

  This holds for Gemma but not Llama.

• p. 24 Figure 9c legend

  Llama-3-70B (r=-0.62, p=0.268)

Judge's note: Read the Figure 9c legend on page 24: Llama-3-8B r=0.29 p=0.527, Gemma-3-12B r=-0.69 p=0.085, Gemma-3-27B r=-0.75 p=0.031, Llama-3-70B r=-0.62 p=0.268. The family-level split in the caption fits neither sign (70B is negative) nor significance (Gemma-12B is not significant either). Appendix J text does not qualify the claim. Wording overstates the data.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send

• p. 25 Table 17, threshold 0.3 (default)

  59.3%

• p. 25 Table 16, all rows

  Total

• p. 25 Table 17, row 0.3 (default)

  0.3 (default) 59.3% 3.7% 9.0% 28.0%

• p. 25 Table 16

  Obf. policy

Only the authors can say what is right here.

Judge's note: Summed every row of Table 16 from the page 25 image: honest 196, blatant 26, obf. policy 65, obf. activations 96, row totals 383 (the four classes also sum to 383). Shares differ from Table 17's default row by up to 8 points and obf. policy is about half. Excluding the alpha=0 rows (306 runs) gives 58.5% / 3.6% / 19.0% / 19.0%, which also does not match. Searched Appendix N, the Table 17 caption and Section 4.4 for a statement of which runs Table 17 covers (e.g. MBPP probe or alpha=100 runs included): none found. A different population is the likely cause, so this is a query on the unstated population rather than a demonstrated wrong value.

Second check: another model agreed

Report a problem with this finding

This finding is

wrong right unclear or overstated already corrected (erratum, later version)

I am an author of this paper Send