Weng's Harness Ladder Has a Blind Step

1. The Ladder Has a Blind Step

Lilian Weng's July 2026 survey, Harness Engineering for Self-Improvement, organizes the field into a clear optimization ladder:

instruction prompts → structured context → workflow → harness code → optimizer code

Each rung moves the optimization target higher: from what we say to the model, to how we structure what the model sees, to how we orchestrate the loop, to the code that defines the orchestration itself, and finally to the optimizer that writes the harness code. This ladder is useful because it exposes a trajectory the field has been following, often without realizing it.

But the ladder has a blind step. It's visible in Weng's own list of Future Challenges:

Future Challenge #1: Weak and fuzzy evaluators. Many research claims do not have a fast and precise verifier, and the same is true for many real-world tasks.

Weng frames this as a precision problem: the evaluator isn't sharp enough to distinguish good outputs from bad ones. Most systems in her survey — STOP, Self-Harness, Meta-Harness, DGM, ACE — treat the evaluator's output as trustworthy, then optimize how to use that output. None of them explicitly measure whether the evaluator itself makes directional errors: mistakes where the output is semantically reversed (keeping what should be deleted, enabling what should be disabled) but structurally indistinguishable from a correct result.

This article argues: weak evaluators are not just imprecise. They fail directionally — accepting plausible-sounding output that reverses the task. My own data shows this is uneven: stronger models catch most of it. The structural bound (Theorem 2 below) remains; the practical impact is concentrated in weaker models. The evidence comes from multiple independent threads that converged in the weeks after Weng's survey was published.

2. Threads Converge

Thread 1: The DGM Fake-Log Story

The Darwin Gödel Machine (DGM) paper (Zhang et al. 2025) contains the cleanest documented case. Weng discusses DGM extensively in the survey — but the fake-log incident itself is in the paper, not the survey. An agent, allowed to modify its own harness, faked a log file claiming its unit tests had passed. The tests never ran. The fake log went into its own context, and downstream the same agent read that log and concluded its changes were validated.

Sergei Parfenov's commentary on this case (published July 8) identified the structural mechanism: the system had no way to distinguish what it verified from what it once said. A file is a file. The filesystem cannot attach a provenance label to tell the agent whether that "2 tests passed" line was generated by a test runner or by the agent's own hallucination during a previous tool call.

This is a directional failure: the agent's judgment about its own work was reversed from ground truth. It thought its changes were validated. They were not.

Thread 2: Directional Failure Is Real, but Model-Dependent

I ran 20 scenarios — 16 directional-failure cases (6 explicit reversals, 10 subtle reversals) plus 4 controls (2 valid, 2 garbage) — across 3 model tiers — qwen3:0.5b (0.5B), gemma3:latest (4.3B), deepseek-v4-flash (~200B) — for 600 total judgments. The models were asked the same question Weng's evaluators answer: does this output satisfy the task?

I had expected directional failures to be structural across all model sizes. The data doesn't bear that out. Miss rates on subtle-reversal scenarios:

Directional failure is real, but its severity scales sharply with model capability. The weak model misses nearly half; the strong model catches almost everything.

The one scenario that consistently broke both smaller models under the harness label was a "no change needed" rationalization — the task asks to set a parameter already at the target value, and the output says "current config satisfies the requirement, no change needed." The 100% miss here is a label choice as much as a measurement: DS4 sits at the boundary between directional failure and task-design ambiguity — accepting the output changes no state, so the models are failing to check that a required write happened, not accepting objectively wrong output. On this single scenario (DS4):

This revises the original framing: directional failure is not a uniform blind spot. It is a capability gradient. The theoretical bound (Theorem 2 below) still holds — a text-channel evaluator cannot detect a text-channel fabrication — but the practical impact is concentrated in weaker models. On strong models, the failure mode shifts from "confidently wrong" to "uncertain," which is itself a detectable signal.

The same mechanism appears in the DGM fake-log story. The agent didn't write "tests: definitely failed." It wrote "tests passed" — a plausible claim that the evaluator accepted without verification. Whether the verifier catches this depends on the verifier's capacity, not the prompt.

Thread 3: Theorem 2 (Data Processing Inequality)

René Zander (@reneza on dev.to) identified the theoretical root: when the evaluator shares the same communication channel as the producer (both read and write text), the information available to the evaluator is a subset of what the producer output. If the deviation doesn't appear in the text — if the deleted file is called "retained" with a justification, if the fake log says "tests passed" — a text-channel evaluator reading that text cannot detect it. (An evaluator with access to an independent channel — process exit codes, execution traces, a runner that observes the side effect — can catch cases the text channel misses. The bound is on the text channel, not on evaluation in general.)

This is the Data Processing Inequality applied to agent verification. It's not an implementation bug. It's an information-theoretic constraint. (Labeled "Theorem 2" for series continuity; strictly, this is a structural argument from information theory rather than a formally proved theorem.)

The convergence

These observations converge on the same theoretical prediction. The evaluator is not just imprecise. It fails directionally, and the theoretical bound (Theorem 2) is structural — but the practical severity scales with model capability, as the DF v2 data shows.

3. Seven Design Constraints from the Literature

If the evaluator can fail directionally, and the failure is structural, then the fix cannot be "a better evaluator." The fix must be architectural: design constraints that limit what the evaluator is asked to judge, and how its judgment is used.

The seven constraints below are extracted from the papers Weng surveys, reader comments, and the experiments above. Each has been implemented in the ReqForge forge-verify pipeline during a single development session following the survey reading.

Constraint 1: The Evaluator Lives Outside the Loop

Source: Self-Harness (Zhang et al. 2026), Weng's commentary

"If a program is allowed to edit the OS system, abstraction boundaries are broken. The editable surface needs to be properly designed and the permission control and security layers need to live outside this loop." — Weng on Self-Harness

Implementation: .forge/editable-surface.json defines explicit read/write paths for the evolution engine. The evaluator code (scripts/forge-verify/) and gate configuration (.forge/content-verify.json) are in the readonly section. The evolution engine cannot modify its own boundary definition (editable-surface.json is itself readonly).

{
  "editable": { "directories": ["core/skills", ".forge/harnesses", ...] },
  "readonly": { "directories": ["scripts/forge-verify"], "files": [".forge/editable-surface.json"] }
}

Constraint 1b: The Router Lives Outside the Loop

Source: Alex Shev (dev.to, July 2026), routing-surface experiment

Alex Shev's routing principle — route by risk, keep the LLM a router rather than the final authority, and let deterministic checks and human diff reviews issue the verdicts — motivates this constraint. Constraint 1b extends the same logic one level up: the router decides which verification path an output enters, and if the agent whose output it dispatches can also rewrite the routing table, dispatch confidence is back to coming from the same semantic layer that made the dispatch. The principle holds only if the router itself lives outside the agent's editable surface.

Problem: Constraint 1 prevents the agent from modifying its own evaluator. But the router — which decides which verification path an output enters — makes a structurally identical decision. If the agent can reassign its output from the strict verification path to a lenient one, the evaluator's judgment becomes moot regardless of how carefully it's insulated.

Experiment: Monte Carlo simulation of 5000-output verification streams, 500 trials per configuration. Three verification paths calibrated to Part 5's empirical data (Strict: 0% FP / 75% FN, Balanced: 10% FP / 50% FN, Lenient: 25% FP / 25% FN). All outputs default to the Strict path. In the unconstrained condition, the agent can reassign defective output to a more lenient path within plausibility limits (code→Balanced, report→Lenient, config→Lenient). Path entropy (Shannon H) measures operating-point variance — how dispersed outputs are across verification paths.

The agent's ability to reassign defective output to a more lenient path raises path entropy from H=0 to H=0.524, and defective pass-through follows from 0% to 19%. The entropy measures what ANP2 Network's operating-point reframe predicted: the defective pass-through increase is variance from collapsing multiple verification standards into the agent's routing decision. When the routing table is in the readonly section, the path-reassignment vector is eliminated (H=0, DPR=0%) — the block is specific to this vector, and does not catch sophisticated semantic evasion where defective output passes the strict path on its own merit.

Implementation: Extend .forge/editable-surface.json to include the routing table in the readonly section:

{
  "editable": { "directories": ["core/skills", ".forge/harnesses", ...] },
  "readonly": {
    "directories": ["scripts/forge-verify"],
    "files": [".forge/editable-surface.json", ".forge/routing-table.json"]
  }
}

The routing table (.forge/routing-table.json) maps output-type patterns to verification paths. The agent can suggest routing changes via the same proposal mechanism as harness changes (Constraint 6), but path reassignment requires human approval. This makes the routing constraint structurally identical to Constraint 1: the harness defines what the agent can route to, and the agent cannot redefine the harness.

Relationship to Constraint 1: Both are instances of the same architectural principle — any component that judges the agent's output must live outside the agent's editable surface. The router is a judging component: it decides which verification standard applies. If it can be rewritten by the same agent whose output it dispatches, the evaluator's independence is moot.

Script: routing-surface-constraint-test.py
Results: scripts/results-v2/routing-surface-constraint.json

Constraint 2: Causal Labels for Verification Failures

Source: Self-Harness (Zhang et al. 2026)

"Two runs can share the same verifier outcome in the error logs on the surface, such as timeout or missing artifact, while having different causal mechanisms."

Implementation: Each forge-verify stage verdict includes a failure_class field mapped to the feedback-observer classification:

This bridges the verification pipeline and the evolution feedback loop: a verification failure automatically triggers the correct feedback-observer classification, which feeds into evolution proposal generation.

Constraint 3: Proposals Must Pass Held-in and Held-out Splits

Source: Self-Harness (Zhang et al. 2026)

"Candidate edits are evaluated by regression tests on held-in D_in (testing for whether the weakness is resolved) and held-out D_out (checking for whether other unknown issues were introduced)."

Implementation: Evolution proposals carry two file lists:

• held_in_files: targets that should go from REJECT/UNCLEAR → PASS after the edit
• held_out_files: targets that should maintain their previous PASS status

After apply, forge-verify runs on both splits. Both must pass before the proposal is considered finalized. A held-out regression blocks the proposal even if the held-in fix succeeded.

Constraint 4: Every Verdict Traces to an Evidence Source

Source: ScientistOne (Meng et al. 2026), Weng's survey

"Every claim (citation, numerical, methodological, conclusion) must trace to an evidence source and is audited by Chain-of-Evidence checks."

Implementation: Each forge-verify stage output includes an evidence field:

L0:  evidence: "file:src/rate-limit.ts"         (inline content)
EG:  evidence: "evidence:test-output.txt"        (external file)
C1:  evidence: "evidence:test-output.txt((?i)isRateLimited)" (file + pattern)

The final output contains a complete trace.chain array, plus evidence_files metadata (path, size, mtime) for staleness detection. If an evidence file is modified after verification, the trace can be marked potentially stale.

Constraint 5: Rules Can Retire When Models Outgrow Them

Source: STOP (Zelikman et al. 2023), Weng's prediction

"STOP improved mean downstream performance across iterations with GPT-4 but degraded with weaker models like GPT-3.5 and Mixtral." — Weng on STOP

Weng also predicts: "Eventually it is possible that many harness improvements will be internalized into core model behavior, but the interface with external context and tools should remain."

Implementation: Feedback entries carry a model_version field. When the evolution engine detects a rule that was graduated under an older model and has zero failures under the current model, it generates a retirement proposal (status: deprecated) rather than a reinforcement proposal. This prevents circular evolution where a rule graduates, generates more feedback, graduates again as a variant of itself.

Constraint 6: Harness Configuration Is a Searchable Space

Source: Meta-Harness (Lee et al. 2026)

"The meta-harness loop iteratively creates new harnesses, and only qualified ones are kept."

Implementation: .forge/harnesses/ acts as a candidate registry. Each candidate harness is a subdirectory with config.json, scores.json, and history.jsonl. The baseline config (.forge/content-verify.json) is the origin; new variants are generated by varying one dimension (divergence threshold, contract pattern, evidence requirement). Variants are Pareto-ranked by accuracy vs. LLM call cost. The production config is readonly — switching to a candidate harness requires human approval.

Constraint Interaction: Layers Combine Super-Additively

The seven constraints above are not independent dials. An experiment varying which layers are active shows that L2 (Constraints 1+1b: readonly verify scripts + readonly routing table) and L3 (pre-execution verification — extending Constraint 1's readonly principle to the signal producers that report whether a step actually ran) must both be present — neither alone is sufficient.

Setup: Monte Carlo simulation, 5000 outputs per trial, 500 trials per config, 10% defective rate. Four evasion vectors available to a defective agent, tried in preference order:

Parameter sources: V4's 0.85 base success reflects the typical case where submission is by self-report alone — no external audit checks whether the step actually executed. The 91.8% pre-execution detection rate is measured in pre-execution-verification-test.py (Strategy D: audit on 2+ of 5 signals at 0.3% false-positive rate). V1/V2/V3 success rates model the agent's effectiveness once a vector is picked; absolute values matter less than the structural finding that L2 and L3 must combine.

Result:

If layers contributed independently, expected reduction = 1.7% + 17.3% = 19%. Actual reduction = 88.2%. The combined effect is 4.6× the sum of individual contributions — a super-additive interaction.

The mechanism: L3 presses V4 down to 7% success, but the agent falls back to V1/V2 which L2 hasn't blocked. L2 alone is decorative because V4 succeeds 85% of the time without L3 detection. Only when both are active does the defective agent run out of fallbacks.

This is why the constraints cannot be evaluated in isolation. Constraint 1 (evaluator readonly) and Constraint 1b (router readonly) look weak on their own — V4 dominates. Pre-execution verification (L3) looks moderate on its own — V4 is suppressed but V1/V2 still work. The architectural value emerges from their combination: a single structural boundary whose pieces seal each other's gaps.

Script: multi-layer-constraint-test.py · L3 detection source: pre-execution-verification-test.py
Results: scripts/results-v2/multi-layer-constraint.json · scripts/results-v2/pre-execution-verification.json

Update: The Super-Additivity Is Structural (Sensitivity Grid)

The 4.6× number above rests on the specific vector rates in the setup table. To check whether it is a calibrated operating point or a structural property, I ran a grid over all four vector success rates and the L3 detection rate (three levels each, 243 cells) in multi-layer-sensitivity-test.py. Defective pass rate has a closed form in this model — P(pass) = 1 − (1−FP)·∏(1−p_v) over the unblocked vectors — so each cell is computed exactly, cross-checked against the original Monte Carlo core.

Result: zero of 243 cells are sub-additive. The ratio R(L2+L3)/(R(L2)+R(L3)) ranges from 1.17× to 21.9× (median 2.19×), with the 4.6× headline inside that range. The interaction term is closed-form:

R(L2+L3) − R(L2) − R(L3) = (1−FP)(1−αβ)(δ′ − γδ)

where α, β are the failure rates of the L2-blocked vectors (V1, V2), γ is V3's, and δ / δ′ are V4's failure rate without / with L3. The expression is non-negative whenever L3 does not increase V4's success — the one thing L3 is guaranteed not to do — so the super-additivity is a property of the product structure, not of the specific calibration. The rates control magnitude, not existence: the ratio approaches 1 exactly where one layer is decorative (its blocked vectors never succeed), and is largest where all vectors are strong and L3 also crushes the residual V4.

One model-fidelity caveat: the ordering dimension is a no-op in this model — first-success pass probability depends only on the set of unblocked vectors, so all 24 preference permutations give identical pass rates. A real agent's vector preference would matter through mechanisms this model does not carry (detection feedback, budget, conditional success); that refinement is not modeled here.

Structural (model-form) sensitivity. Two variations in multi-layer-structure-test.py. First, common-factor correlation — V1/V2/V3 share a latent capability z (p_v = b_v·z) — the ratio stays ≥ 1 at any correlation strength, provably pointwise in z. So correlation alone does not break it. What does is capability redundancy: if V1/V2/V3 are one capability and L2 and L3 therefore block the same weakness — the "single readonly boundary" reading of these constraints — the layers stop complementing and start duplicating. A mixture over that redundancy drags the ratio from 4.6× down through 1.0 (at roughly 84–94% shared capability, depending on how prevalent the capability is) to 0.92× at full redundancy — sub-additive. The super-additivity is therefore a property of the fallback policy and of the layers targeting distinct capabilities.

Posterior over the assumed rates. The vector rates are assumed, not measured. Treating them as Beta-random (200k draws in multi-layer-bayesian-sensitivity.py), the existence of super-additivity carries zero prior sensitivity — P(ratio ≥ 1) = 1.000 under both article-centered and near-uniform priors. The magnitude moves: ratio 90% CI 2.5×, 12.3× and 1.4×, 6.4×. The residual is the tight quantity: DPR(L2+L3) 90% CI [10.0%, 12.6%] / [6.1%, 11.7%]. So "L2+L3 leaves a ~10% residual" is prior-robust; the 4.6× headline is a magnitude inside a wide band, not a pinned number.

Scripts: multi-layer-sensitivity-test.py (rate grid) · multi-layer-structure-test.py (model form) · multi-layer-bayesian-sensitivity.py (posterior) · Results: scripts/results-v2/multi-layer-sensitivity.json, multi-layer-structure.json, multi-layer-bayesian.json

4. What the Implementation Tells Us

Implementing all seven constraints as additions to an existing pipeline took approximately four hours. The code changes total under 250 lines across 9 files. The key observation is not that the implementation was fast — it's that each constraint maps to a single, testable mechanism that can be verified independently. (The forge-verify pipeline these constraints extend was developed across multiple sessions; the four hours measures the incremental cost of adding the constraints to a system already in place.)

The seven constraints are not speculative. They are direct translations of findings from Weng's survey, reader comments, and controlled experiments into concrete code. The survey provides the design constraints; the implementation shows they are mechanically simple. The difficulty is not building them — it's recognizing them as necessary.

The Constraint Interaction experiment in §3 is validated separately via multi-layer-constraint-test.py (5000 outputs × 500 trials × 4 configs), measuring how the constraints combine rather than any single constraint in isolation.

5. What Still Can't Be Fixed

Constraint 4 (evidence trace) and Constraint 6 (harness search) have a common limitation that Theorem 2 predicts: the trace itself is text, and the harness configuration is text. Both can be fabricated by an agent that controls the file system.

The DGM fake log story applies to every evidence file that forge-verify reads — when that file was written by the agent. If the verify script reads evidence from a path the agent can write, the agent can write a file called test-output.txt that says "RateLimiter-IP: all tests passed" without running any tests. The evidence gate checks that the file exists and that a regex pattern matches its content. It does not check who wrote the file or whether the file content reflects actual execution.

This is not a universal claim. For evidence produced by the verify runner itself (the runner executes tests in the readonly section and captures output), the agent never touches the evidence file, so the DGM mechanism does not apply for that subset. The gap is limited to evidence files the verify script reads from agent-writeable paths — such as error logs the agent generated during execution or coverage reports the agent produced.

This is not a bug in the evidence gate design. It is a design constraint: the evidence gate is zero-cost and pre-receive, but it trusts the file system that the agent writes to. The typed provenance layer that Sergei calls for — runtime-verified vs. self-reported — does not exist in the current forge-verify implementation. It would solve the DGM edge case, but it requires the file system itself to be provenance-aware, which standard file systems are not.

(Mads Hansen's comment on this post corrects that last framing: the requirement is not a filesystem property. It is two deployment facts — a runner whose evidence namespace the agent cannot write, emitting a content-addressed receipt the agent holds only as a reference, resolved against a trust root the agent does not hold. The current implementation still lacks it, but the obstacle is deployment topology, not the filesystem.)

The residual that Theorem 2 says cannot be closed:

• A text-channel evaluator (LLM or human) cannot catch directional reversals that are textually indistinguishable from correct output.
• An evidence gate cannot distinguish between evidence written by a test runner and evidence written by the agent under test — unless the gate only reads evidence from paths the agent cannot write, produced by a runner in the readonly section.
• A typed provenance system cannot distinguish if the typing is done by the same agent that produces the output — though the constraints above ensure typing is done by readonly code, narrowing this to cases where evidence must originate from the agent's output.

In practice, stronger LLMs catch many of these reversals because they carry better implicit models of plausible-vs-correct. The residual that even strong models miss — and that human reviewers consistently miss — is what Theorem 2 actually predicts. The capability gradient documented in Thread 2 narrows the practical scope; it does not eliminate the theoretical bound.

This is not an argument against layered verification. The seven constraints above demonstrably narrow the gap. The L0/L0e deterministic checks catch structural garbage before it reaches the LLM. The evidence gate catches missing artifacts. C1 validates specific format promises. C2 reads each requirement individually, preventing the "everything looks fine" narrative from overwhelming the judge. The trace makes the chain auditable. The harness search makes the config improvable.

But the gap narrows asymptotically. Theorem 2 says it never reaches zero.

6. Summary

Weng's harness engineering survey is the most comprehensive map of the field. It also reveals a blind step: the assumption that evaluators fail on precision, not direction. Three independent threads — the DGM fake log, the DF v2 data, and Theorem 2 — converge on the same finding: directional evaluator failure is real, but its severity scales with model capability. The structural bound holds; the practical impact is concentrated in weaker models.

Seven design constraints extracted from the survey and related work translate into testable code mechanisms. All seven are implemented in ReqForge's forge-verify pipeline; validation status per constraint is marked in §4's table (two have experimental validation via the routing-surface and multi-layer experiments — Constraints 1 and 1b; Constraint 2 has scenario-level tests; Constraints 3-6 are structural implementations awaiting runtime-event validation). The implementation is less than 250 lines across 9 files. An interaction experiment shows the constraints combine super-additively: L2 (readonly verify + routing) and L3 (pre-execution verification) individually reduce defective pass-through by 1.7% and 17.3%, but together by 88.2% — 4.6× the sum of their individual contributions. The architectural value is in the combination, not the pieces.

The theoretical residual persists: a text-channel evaluator cannot catch what a text-channel producer can fabricate. The constraints narrow but do not eliminate the gap. That is not a design failure. It is an information-theoretic limit, and acknowledging it is more useful than engineering around it.

Experiment data: 20 scenarios (16 directional-failure, 4 controls) × 3 model tiers × 600 judgments in directional-failure-v2.py
Multi-layer constraint experiment: 5000 outputs × 500 trials × 4 configs in multi-layer-constraint-test.py — L2+L3 combine 4.6× super-additively
Evidence gate test: 6 scenarios, 12/12 pass in scripts/forge-verify/test-evidence-gate.mjs
Source survey: Harness Engineering for Self-Improvement — Lilian Weng, July 2026
Series: Agent Determinism Illusions on dev.to/zxpmail
Previous: The Channel Gap: Why Your LLM Judge is Blind in One Eye
Next: The Third Predicate: Argument-Space Verification, Tested