Skip to content

DEV Community

Agent Determinism Illusions Series' Articles

Back to zxpmail's Series

Formal code failing at the semantic layer

zxpmail

Jul 4

I tested the 'deterministic agent loop' claims with four experiments. They all failed — including my own fix.

#ai #llm #agents #testing

9 min read

Precision-recall and the false positive mirage

zxpmail

Jul 5

I tested 3 models as AI agent quality inspectors: the stronger the model, the more valid work it rejects

#ai #llm #testing #agents

5 min read

zxpmail

Jul 6

I designed a Harness to fix my agent's quality problem — then found 6 flaws in my own design

#ai #llm #testing #agents

8 min read

Six experiments on LLM-judge failures

zxpmail

Jul 9

An alternative to LLM quality gates: deterministic routing + sampling

#ai #llm #agents #testing

11 min read

zxpmail

Jul 14

Six experiments on adversarial verification — and the 75% wall that didn't move

#ai #llm #agents #testing

6 min read

zxpmail

Jul 18

The Red Line Principle: objective stop signals outperform LLM self-judgment in verifiable tasks

#ai #llm #agents #testing

13 min read

zxpmail

Jul 21

Five Comments That Redesigned My LLM Verification Pipeline

#ai #llm #agents #testing

26 min read

zxpmail

Jul 23

Divergence escalates the wrong population: unanimous misses auto-pass

#ai #llm #agents #testing

17 min read

Rigorous self-correction via 600 test calls

zxpmail

Jul 25

I Fabricated a Claim About LLM Judges. Then I Ran the Apology Experiment.

#ai #llm #agents #testing

8 min read

zxpmail

Jul 28

D+T2 names who enters; budget names who gets seen

#ai #llm #agents #testing

6 min read

zxpmail

Jul 30

Reader-driven revisions: four comments that bit back

#ai #llm #agents #testing

5 min read

zxpmail

Aug 4

Round 2: when the reply triggers another revision

#ai #llm #agents #testing

4 min read

Combines text judges with filesystem checks

zxpmail

Aug 6

The Channel Gap: Why Your LLM Judge is Blind in One Eye

#ai #llm #agents #testing

15 min read

zxpmail

Aug 11

Weng's Harness Ladder Has a Blind Step

#ai #llm #agents #testing

18 min read

zxpmail

Aug 13

The Third Predicate: Argument-Space Verification, Tested

#ai #llm #agents #testing

15 min read