I spent a while optimizing what my AI agents cost to run.
Cheap models for routine work. Better models when the task actually needed them. That made it practical to run several agents without sending every request to the most expensive model available.
It worked. Then I hit the next bottleneck: trusting what came back.
An agent can say a task is done, show passing tests, and leave a clean-looking diff. None of that tells me whether it understood the product decision, changed the right files, or missed an assumption elsewhere in the codebase.
More agent capacity did not remove that problem. It helped me create it faster.
I now use Sol Advisor with Codex to separate architecture, implementation, and review. I keep the objective, architecture, verification, and final acceptance. The implementation agent gets a bounded job. A fresh reviewer challenges the result.
The workflow
define the change
-> choose the implementation lane
-> delegate a bounded task
-> inspect the diff and rerun checks
-> get a fresh review
-> ship, fix first, or rethink
The implementation report is a claim, not proof. I inspect the actual working tree, confirm the task stayed in scope, and rerun the promised checks before asking for a review.
That procedure is the part I was missing. Model routing answers, "What should this task cost?" Sol Advisor helps answer, "Should this patch ship?"
Install Sol Advisor
You need Codex and Bun installed. Then add the repository marketplace and install the plugin:
codex plugin marketplace add DannyMac180/sol-advisor --ref main
codex plugin add sol-advisor@sol-advisor
Start a new Codex task and invoke the workflow:
Use $sol-advisor:orchestration for this task. Verify the implementation and obtain the configured advisor review before reporting done.
The first run walks you through setup. Sol Advisor's repository contains the current installation notes, while the orchestration skill shows the exact routing, verification, and reviewer contract.
Installation gives you the workflow. The quality of the handoff still depends on what you give it.
Copy my work packet
I do not delegate with, "Make this page better." That is how you get a polished patch that solves a problem nobody asked for.
I paste this into the parent task and fill in every section:
# Objective
[What observable outcome should exist when this is finished?]
# Scope and ownership
- May change: [owned files or responsibility]
- May inspect: [related files]
- Out of scope: [explicit exclusions]
- Preserve unrelated edits already in the repository.
# Interfaces and constraints
- [behavior that must remain unchanged]
- [types, APIs, design rules, or security limits]
# Verification
- Run: `[exact command]`
- Manually confirm: [behavior and edge case]
# Evidence expected
Report the files changed, checks run, assumptions made, and anything still uncertain.
The packet takes a few minutes to write. It saves me from reviewing a technically valid solution to the wrong problem.
Choose the lane by risk
| Task | How I run it |
|---|---|
| Typo, label change, or obvious one-line fix | Handle directly |
| Bounded UI work, documentation, wiring, or mechanical refactor | Routine implementation |
| Authentication, concurrency, migration, public API, difficult debugging, or wide refactor | High-complexity implementation, with architecture review before code when needed |
| Independent tasks with separate files and no shared contract | Run in parallel |
| Tasks sharing files or changing an interface the next task consumes | Run sequentially |
Agent count is not a reason to create a merge conflict. If two tasks cannot own separate responsibilities, I keep them in order.
Verify before asking for review
When the implementation comes back, I check five things:
- Inspect the complete working-tree status and diff.
- Confirm only in-scope files changed.
- Read the code instead of trusting the summary.
- Search for consumers of changed interfaces.
- Rerun the checks from the work packet.
Then I give a fresh reviewer this smaller packet:
Review this implementation without modifying the repository.
Objective: [paste objective]
Scope: [paste owned responsibility]
Constraints: [paste relevant constraints]
Evidence: [paste checks and observed results]
Inspect the actual diff and relevant surrounding files. Check correctness,
scope, hidden consumers, regressions, missing tests, and whether the evidence
proves the objective.
Return one verdict:
- ship: no blocking issue remains;
- fix-first: list the smallest required corrections;
- rethink: the implementation or underlying approach is wrong.
Separate blocking findings from optional improvements. Do not implement fixes.
If the verdict is fix-first, I send the correction back to the implementation lane, rerun the checks, and request another fresh review. Any changed diff invalidates the old verdict.
Where this is slower
I do not use this entire loop to rename a variable. The overhead makes sense when a task has ambiguity, hidden consumers, or enough blast radius that rework would be expensive.
It also cannot rescue a vague product decision. If I cannot explain the outcome and constraints, multiple agents will produce a more elaborate version of my confusion.
The actual productivity gain
The scarce resource is no longer generated code. It is the attention required to decide whether that code belongs in the product.
This workflow lets me spend that attention deliberately. I keep the decisions and acceptance bar. The implementation agent gets a job it can finish. The reviewer gets a clean chance to disagree.
Try the packet above on one medium-sized task. Keep the scope tight, inspect the diff yourself, and ask a fresh reviewer what the implementation missed.
When the agent says it is done, you should have a better answer than, "The summary sounded convincing."
Top comments (8)
Model routing as cost optimization makes sense in isolation, but you have identified the real tradeoff. Cheaper models produce more outputs that need verification, and verification is the expensive input nobody budgets for. The total cost curve probably has a U-shape. Past a certain point, routing to cheaper models increases your verification surface area faster than it reduces your inference spend.
The fresh-review step gets much stronger if “fresh” also means unanchored. I’d have the reviewer first derive risks and acceptance checks from the objective/constraints, then reveal the implementation diff and its report. Otherwise the implementer’s framing can quietly define what the reviewer looks for. Bind the verdict to a base commit plus diff hash, and add one adversarial check for the task class—mutation test, rollback/upgrade path, concurrency race, or a deliberately broken fixture that must fail. Over time, track escaped defects and false-
shipverdicts by task class and model route; that turns trust from a workflow intuition into something you can calibrate.The “implementation report is a claim, not proof” line is the key lesson.
I’d treat trust as a provenance problem more than a routing problem. The useful record is not only which model answered, but what files it inspected, what tools it used, what checks actually ran, which assumptions changed, and what the reviewer saw before giving a verdict.
Routing can reduce inference cost, but provenance is what keeps the cheap path from turning into invisible risk.
The "implementation report is a claim, not proof" line matches something I keep re-learning the hard way. I run a fairly strict verify-from-clean-state discipline on an MCP server project — every fix gets a status of "verified" or "⚠️ not verified" in the changelog, and the ⚠️ ones vastly outnumber the confirmed ones even when the agent's own summary reads as done.
The most useful version of your reviewer step, for me, has been asking a second agent to specifically try to disprove the first one's claim rather than just review the diff — one item marked "fixed" in an earlier pass turned out to be a false positive once actually re-tested from a clean checkout. Cheap to catch, expensive to have shipped.
Dean's U-curve point below is the sharper way to put it: cheaper implementation is only a win if verification cost doesn't eat the savings, and a "sounds done" summary is exactly the kind of evidence that hides that cost until later.
The "implementation report is a claim, not proof" line is the whole thing for me. I hit the same wall: agent says done, tests green, diff looks clean, and none of it tells you whether it understood the decision it was making.
What I keep circling on: your review step is still a human (or a fresh model) reading the output after the fact. Has anyone managed to push that check down into the run itself, so the claim gets verified against what actually executed instead of what the agent says executed?
Also curious how the fresh-reviewer pattern holds up over time. In my runs the reviewer starts absorbing the same assumptions as the implementer after a few rounds, and the "fresh" part quietly disappears.
The lane nobody logs is the review lane. If routing picks a model per task, your fresh reviewer got routed too, and a cheap reviewer will happily sign off on a bounded diff that quietly broke an assumption two files over. I'd pin the reviewer to a fixed model and record which model ran each stage, so when something ships wrong you can tell a lane failure from a model failure. We landed on something similar with Viewfy, where a human approves every reply before it posts: the gate only counts if what you're approving is small enough to actually read in one pass.
Cheaper but not more trustworthy is a tradeoff people underestimate. Routing easy queries to a smaller model quietly changes your quality distribution. I started logging which model answered each request so I could attribute failures back to a routing decision. Are you setting the threshold on a confidence score or on query classification?
The claim vs proof distinction hits hard. Green tests and a clean diff still hide cases where the agent solved a different problem than the one I actually needed fixed.