If you think logging your LLM responses is enough for compliance, this post is for you.
August 2026. The EU AI Act enforcement begins for high-ris...
For further actions, you may consider blocking this person and/or reporting abuse
A per-run hash chain gives you tamper evidence over the receipts that exist; completeness is a separate property and
previous_hashdoes not carry it. The signer here is also the audited party:signatureproves a key claimed a step, andstepis a counter chosen inside arun_idthe same key minted. So the cheap attack is not forgery. Under pressure a runtime can close one run and open another, and the inconvenient run leaves no numbered hole anywhere. What closes that is a monotonic sequence scoped to the signing key rather than to the run, so a discarded run shows up as a gap, plus a tool-side enforcement layer holding its own key and its own counter and countersigning what it actually executed. Audit then becomes reconciliation between two independently kept chains, and the discrepancy is the finding. A single self-signed chain has no discrepancy available to it.With the stale-state gap covered upthread,
reasoningis the field I would be most careful about signing. It is model-generated prose about model behavior, so signing it takes an unverified narrative and gives it the shape of evidence, when the signature only attests custody. The drift-detection gap you name sits downstream of the same problem: the model is free to vary how it narrates an identical decision, so measuring drift against that field measures style as much as judgment. It also runs straight into WORM. Free-text model output is where pasted secrets and personal data land, and Object Lock exists precisely so nothing can be removed during the retention window, which puts Article 12 retention and an erasure request in the same field with only one possible winner.Cheaper pattern: commit
reasoning_hashin the receipt and keep the prose in ordinary deletable storage addressed by that hash. The chain still proves the text existed and has not changed, and the text can still be deleted. Structured decision facts go in the receipt as fields: rule fired, inputs consulted, alternatives rejected, and a policy content hash rather than justv2.3.1, since a version string can be rebuilt under the same name.Self-signed chain and run grafting
You're right: if the same key mints run_id and chooses the step counter, a compromised runtime can drop a run and start another without leaving a hole. A proper implementation should enforce a monotonic sequence scoped to the signing key across runs, so any gap in step numbers becomes detectable.
Tool-side countersigning
The two-chain reconciliation model you describe is exactly the right target: one side signs what the agent claims, the other signs what the tool actually executed. Without that, you have no discrepancy to detect.
reasoning as signed evidence
This is the most important point. Signing model-generated prose gives it the shape of evidence while only attesting custody. That's actively misleading in an audit context. The right move is to store reasoning_hash in the receipt, keep the prose in ordinary mutable storage addressed by that hash, and put structured decision facts in the receipt itself — rule_fired, inputs_consulted, alternatives_rejected, and a policy_content_hash computed over the actual policy bytes, not just a version string.
WORM vs. erasure
The conflict between retention obligations and erasure requests is real, and putting free-text model output into immutable storage makes it worse. The pragmatic fix is boundary-based: structured audit facts in WORM, free-text reasoning outside it, with the hash linkage preserved.
Thanks for the thorough breakdown.
Reconciliation only produces a finding if the two chains are able to disagree. If one party deploys and operates both the runtime and the tool-side enforcement layer, that is a single trust root holding two keys, and both chains go quiet together under exactly the pressure the audit exists to catch. The useful question to ask about a second signer is whether it has any reason to keep signing at the moment the first would prefer it stopped. That is what decides where the enforcement layer belongs: the far side of a network hop, or a credential domain the audited party does not administer. It also reframes the API limitation raised upthread. When a target offers nothing to check against, what is absent is custody of an independent witness, and no signature scheme supplies that.
The boundary split moves the retention conflict off the immutable side, and it is worth writing down what that costs. Once the prose is deleted, the committed reasoning_hash still shows that something existed and was unaltered, but the preimage is gone permanently, so that field degrades from a record of what was said into a proof that a string of some length once hashed to this value. Probably the right trade. It does mean an erasure request granted years later quietly changes what a future audit can conclude, without touching the chain at all. So the structured facts have to carry the claim unaided, which is a heavier requirement on that field list than it looks. Anything recoverable only from the prose is a field that can disappear.
The receipt is still missing the state the action was based on.
input_hashandarguments_hashpreserve the recorded request, but they do not show whether the target was still in the same state when the tool ran. A signed receipt chain can preserve a stale decision perfectly.For a sensitive action, I would bind the request to a verified target version and stop if that state moved.
I wrote about that gap here: The reasoning was right, but the world shifted.
This is a sharp point, and you're right — input_hash and arguments_hash prove the request, not the world it landed in. A receipt chain can be internally consistent while every decision in it was based on stale state.
The gap you're describing is a TOCTOU at the agent level: the reasoning step happened against state S₁, the tool executed against S₂, and nothing in the receipt captures that shift.
I like the direction you propose — binding the request to a verified target version. In an auditable agent pipeline, that would look like:
The challenge is making this practical without snapshotting the entire world. For file-based agents, it's tractable. For live APIs, you'd need versioned reads or conditional requests, which not all APIs do.
Read your post — the "reasoning was right but the world shifted" framing is exactly the right way to put it. Thanks for the pointer.
The API limitation is exactly where the system has to be honest. The agent can state which version it read, but a trusted component still has to verify that version before the action happens.
Email is a simple example. A new outbound message often has no existing target version to compare. You may be able to bind it to the thread or draft state it was based on, but the send itself remains a new external effect.
If the target API offers no reliable state check, the system cannot honestly claim strong state-bound execution. The enforcement layer has to evaluate which guarantees the target can actually support, not only audit what the agent requested.
That is one of the design constraints behind MCP Boundary: no state-binding claim unless the target gives the enforcement layer something real to verify. If it does not, the action has to be treated as broader instead of pretending the state was verified.
Thanks for reading the post. TOCTOU is exactly the systems term for the gap I was trying to describe. Now we just need to convince every API provider to give us something real to check :D
Are you running agents in production ? How you would handle compliance ?
We do, but I try to keep the evidence layer independent from the orchestration framework as much as possible.
In practice, I treat the agent framework as an execution engine, not as the source of audit truth. Every sensitive action produces its own signed receipt with immutable metadata (policy version, timestamps, actor, input/output hashes and a link to the previous receipt). That way the evidence survives even if the orchestration framework changes in the future.
One lesson I learned is that emergency bypasses also need the same level of discipline. A permanent “temporary” bypass can silently invalidate an otherwise sound security design. Time-limited, fail-closed escape hatches with explicit audit records have worked much better for us operationally.
I think that’s the balance we’re all trying to reach: keeping the audit model framework-agnostic while still making it practical enough that teams actually adopt it.
This is gold. The emergency bypass point is something I should have included in the post.
You're right that the "temporary permanent bypass" is where most audit models collapse. Everyone designs for the happy path — agent proposes, human approves, receipt gets signed. But at 3 AM when something is broken and someone needs to override the policy gate now, that override either gets logged properly with the same hash-chain discipline, or it gets done through a backdoor that nobody documents and the audit becomes fiction.
The time-limited fail-closed escape hatch you describe is the right pattern. It forces the bypass to be intentional and temporary by design, not by policy. The system reverts automatically — no human has to remember to turn the bypass off. And the audit record of the bypass itself becomes part of the evidence chain, not a gap in it.
This is exactly the kind of operational reality that framework vendors don't document. Theory is clean; 3 AM production is messy.
Have you written about your implementation anywhere? The combination of framework-agnostic receipts + disciplined bypass handling would make a great case study.
Thanks, I appreciate that. I haven’t written it up yet, but I probably should. The implementation evolved from solving production problems rather than trying to design a perfect audit model from day one. The emergency bypass was one example where operational reality forced a change in the architecture. I’ll try to document the approach once the remaining rollout work is complete.
Please, keep me posted once you do it. I'd love to hear about real examples outside
Thanks, I definitely will. The discussion here actually changed how I’m thinking about the article. I was originally going to focus on the audit architecture itself, but I think the more interesting story is what production taught us after the architecture was “finished.” The emergency bypass ended up exposing assumptions we didn’t realize we had, and from there we started redesigning things like verification, ADRs, and operational evidence rather than just the receipt format. If that experience is useful to others, it’ll be worth documenting.
If I had to make this auditable, I would separate the evidence layer from the agent framework. Traces are useful for debugging, but audit evidence needs stable event IDs, policy version, approver, external write, and a tamper-evident chain that survives a vendor swap. Otherwise the compliance story depends too much on whichever orchestration UI happened to record the run.
100% agreed. That's exactly why the receipt schema in the post includes policy_version, previous_hash, and a standalone Ed25519 signature, not just whatever the orchestration UI happened to log. The hash-chain survives a vendor swap because each receipt is self-contained — you can replay the chain from any storage backend, not just LangSmith or LangFuse. The hard part we're finding is making this lightweight enough that small teams actually adopt it. Everyone agrees on the theory; the implementation friction is the real barrier. Have you built something like this in production? Would love to hear what evidence layer you landed on.
I'm actually building something like that. V.E.L.O.C.I.T.Y. IDE, which uses a merkle root SiteMap and VC system. Essentially a drift-prevention mechanism, with a full audit trail, including agent context, while also allowing multi-tenancy with realtime discourse handling, instead of arbitrary git-merges.
Auditable is not nice to have feature anymore, its a must feature