
Your deterministic engine ran correctly on the wrong input.
Source says 450000. Model emits 45000. JSON Schema still says “number.” The tool allowlist still says “ok.” Your deterministic engine — rules DAG, ledger, calculator, claims/tax pack, anything that must not invent — runs correctly on the wrong world.
That’s the bleed. Deterministic systems don’t “hallucinate.” They fail correctly on bad inputs. Schema and allowlists never asked whether each argument was bound to a verbatim evidence span.
Old way
Most stacks treat tool calling as:
- Expose the tool
- Validate JSON shape
- Call the engine
Useful. Incomplete.
What they usually can’t do:
- Bind a field to a verbatim evidence span
- Refuse execution when the proposal doesn’t match the document
- Hand the engine a signed authorize-before-execute artifact — not a soft
0.87confidence
Equation:
Deterministic Engine + Unverified Probabilistic Input
= Deterministic Wrong Answer
New way
Put a tool-argument gate between proposal and execution:
- Capture evidence — what the document actually said (spans)
- Ground — tie each field to a verbatim span
- Decide — allow · send to human · or refuse
- Sign — Ed25519 ParameterManifest
-
Enforce — fail-closed hard check (
enforce_group3_boundary) before the engine runs
Probabilistic extractors and agents may still propose. They don’t get to authorize.
That’s PrismManifest (Apache-2.0, prismmanifest==0.3.4) — orchestrator-agnostic. LangGraph, CrewAI, custom Python, C++ engines, your existing DAG. ChorusGraph is optional, not required.
Is / is not
| Is | Is not |
|---|---|
| Trust boundary before a deterministic engine runs | An LLM / full OCR product |
| Evidence → signed manifest → hard check | Prompt-injection firewall (that’s a different product class) |
| Fail-closed | “0.87 confidence” as permission to execute |
Money paths are a sharp case; the same gap shows up anywhere untrusted args enter deterministic compute.
Where the guarantee sits (don’t blur this)
| Stage | Today (0.3.4) | Guarantee |
|---|---|---|
| Span resolve / digit binding | Python | Field ↔ evidence span |
| Plausibility (integer micro-units) | Python | Exact money math — no FP drift |
| Quorum / gate status | Python | allow / human / refuse |
| Signed FlatBuffer enforce | C++ (+ Python twin) | Artifact valid before engine |
C++ protects the signed ParameterManifest at the final boundary. Span binding and plausibility stay in Python today. CUDA is optional acceleration — not the security thesis.
Minimal shape
from prismmanifest import KeyRing, PrismManifestPipeline, enforce_group3_boundary, GateDecision
from prismmanifest.router import DocumentPackage, IntentRouter
keyring = KeyRing.generate(key_id="local-dev-ed25519")
package = DocumentPackage(
doc_id="1040.txt",
pages=["Form 1040 Tax Year 2024\nLine 11 AGI: $450,000.00\n"],
form_type="IRS_FORM_1040",
tax_year=2024,
)
routed = IntentRouter().run(package)
pipeline = PrismManifestPipeline(keyring)
result = pipeline.run_on_evidence(evidence=routed.evidence, extraction=routed.extraction)
gate = enforce_group3_boundary(
result.manifest,
public_keys=keyring,
expected_dag_id="capital_gains_v3",
)
if gate.decision is GateDecision.ACCEPT:
# only then may your deterministic engine run
pass
Install:
pip install "prismmanifest==0.3.4"
# CLI: prismmanifest-gate
# Optional: pip install "prismmanifest[kms-azure]"
Try it on the live site (no upload)
Fixed fixture: Form 1040 · tax year 2024 · Line 11 AGI $450,000. Not your documents.
Digit Drop Lab (10-second punch)
https://www.insightits.com/products/digit-drop-lab.html
| Action | AI proposes | Live badge |
|---|---|---|
| Drop a digit | $45,000 |
Blocked — engine does not run · wrong by $60,750 vs document math |
| Correct extract | $450,000 |
Allowed — engine may run |
Full interactive demo
https://www.insightits.com/products/prismmanifest-demo.html
- Digit-drop → Blocked
- Correct extract → Allowed
- Needs human → Needs human — engine waits
- Custom → enter your own proposed dollar against the same evidence
Pattern tabs (underwriting · insurance claims · accounts payable) show the same money-boundary pattern. Live ACCEPT/REJECT still binds Form 1040 Line 11 AGI $450,000 — those tabs do not run a live DTI / settlement / ERP engine in the browser demo.
Landing: https://www.insightits.com/products/prismmanifest.html
Does the gate actually catch the digit drop?
Pilot OSS vendor harnesses (not a live customer fax SLA):
- Planted money errors: critical FA 0/11
- FinancePackBench text: 100/100 · FA rate 0
- FinancePackBench PDF: 20/20 · FA rate 0
- 500-pack Py × C++ × CUDA decision + signing-hash parity: 511 checks · 0 mismatches
Production bar remains your corpora through the same gate.
Honesty
- Not “beat GPT on OCR accuracy.” Extraction quality ≠ execution authorization.
- Not who-may-speak (injection) and not answer-grounding — different layers.
- Soft confidence is not a substitute for enforce.
- Demo = fixed sample packs · Pilot OSS.
Closing
Deterministic engines will keep doing math correctly. The systems problem is authorizing what they’re allowed to compute on.
Propose freely → sign a ParameterManifest → enforce before deterministic compute.
If your stack still feeds model args straight into an engine, drop a comment with where that crossing happens — or reply MANIFEST and I’ll send an async boundary one-pager (no call).
Top comments (1)
Mine crosses at model output to game state, in a game where AI bots play against people. The guard is type and enum validation at the boundary plus a retry that reruns just that step. It catches malformed. It does not catch well-formed and wrong, and there is no document to bind a span to - the output is not a transcription of anything, so the evidence step has nothing to grab.
On the document side, where do the spans come from on a scan? Your fixture is a text page with the AGI already sitting in it. If OCR produces the proposal and OCR also produced the evidence text it gets checked against, the gate is comparing two draws from the same source.