You retyped the Schematron into PHP. How do you know you got it right?

Last time I wrote about a let variable thirty lines above an assertion that quietly inverted a rule, and made my validator report a bug on every credit note. A few people replied with versions of the same question, and it is the right one:

if you cannot run the official file, and you retyped it into PHP by hand, what tells you the two still agree?

Nothing did. That is what this post is about. The answer turned out to be more useful than the Croatian rules it was built for, so the tool is written to point at any country's ruleset, and you can run it against yours.

The setup, briefly

European e-invoicing rules ship as ISO Schematron. Each country publishes a .sch file layered on top of EN 16931: Croatia's HR CIUS, Germany's XRechnung, Italy, Poland, France. When your invoice is rejected, the rejection quotes an id from that file.

Croatia's declares queryBinding="xslt2". PHP's XSLTProcessor is libxslt, which is XSLT 1.0 only. There is no flag for this. So you have three options: ship a PECL extension nobody can install, make a network call in the middle of issuing an invoice, or reimplement the rules in PHP.

I reimplemented them - 62 rules, one small class each, named after the official id.

That last choice is the one that needs defending, because reimplementation means transcription, and transcription drifts. A regex that is subtly wrong. A rule you read as applying to invoices when it applies to both invoices and credit notes. A let binding you did not notice.

You will not find these by rereading your own code. You wrote the bug by reading the file and you will reread it the same way.

The idea

You cannot run Schematron in PHP. But nothing says the comparison has to run where the validator runs.

So: run the official Schematron somewhere that does have XSLT 2.0, capture what it says about a document, run your PHP over the same document, and diff the two lists of rule ids. Any difference is a bug in the PHP until proven otherwise.

Saxon needs a JVM, which is exactly the dependency I refused to put in the package. But in CI it is free - a container that exists for ninety seconds on a GitHub runner and never touches a user's machine.

# compile the Schematron to an XSLT that emits SVRL
java -cp saxon.jar net.sf.saxon.Transform -s:rules.sch -xsl:schxslt/pipeline-for-svrl.xsl -o:compiled.xsl

# run it over a document
java -cp saxon.jar net.sf.saxon.Transform -s:invoice.xml -xsl:compiled.xsl > report.svrl

SchXslt does the compiling. The output is SVRL, a small XML report where every failure is a <svrl:failed-assert> carrying the rule id:

<svrl:failed-assert id="HR-BR-40" location="/*:Invoice">
  <svrl:text>[HR-BR-40] - Datum izdavanja računa mora biti veći od 01.01.2026</svrl:text>
</svrl:failed-assert>

Pull the ids out, sort them, compare to what your PHP said. That is the whole idea.

The first time I ran it, it found the credit note bug I wrote about last time. That is not a coincidence - it is the category of bug this catches, and the reason I stopped trusting my own reading of the file.

Making it not about Croatia

My first version hardcoded the Croatian .sch in two places. That made "does this PHP agree with the artifact it was transcribed from" a question you could only ask about Croatia, which is silly, because nothing about the problem is Croatian. It is a property of retyping a formal document into a programming language.

So the ruleset became a parameter: an id, a path to a .sch, a corpus, and the implementation under test behind an interface rather than my own validator class.

schematron-diff list
# hr-cius-ext-2025    HR CIUS/EXT 2025
# en16931-ubl         EN 16931 UBL (bundled example)
#                     versions: 1.3.16 (default), 1.3.15

schematron-diff run hr-cius-ext-2025
# 20 agreed, 0 disagreed

Point it at your country's file and your own validator, and you get the same answer about your own code.

Three things had to be right before the output was worth reading, and none of them were obvious to me in advance.

1. Not every assertion is your business

The EN 16931 UBL Schematron has 979 assertions. Only 223 of them are business rules. The other 756 are UBL-CR-*, UBL-SR-* and UBL-DT-* - syntax restrictions saying which UBL elements a conformant document may not use.

If you diff everything against everything, every document reports several hundred "missed" rules and the real result is invisible. So a ruleset declares which id prefixes the comparison is entitled to an opinion about, and everything else is filtered from both sides before comparing.

That sounds like a detail. It is the difference between a report you read and a report you close.

2. A missing rule is not a wrong rule

This is the one I got wrong first, and it matters most if you are comparing against something other than your own complete implementation.

There are two very different findings:

• Disagreement. Both sides have an opinion about this rule id and they differ. That is a bug.
• Gap. The Schematron flagged an id your implementation does not implement at all. That is a coverage hole, and it is not a bug.

Report them the same way and a partially complete implementation looks catastrophically broken, while the actual defects drown in the noise. So the implementation under test declares which ids it implements, and anything outside that list is counted separately and never fails the run.

11 agreed, 0 disagreed

10 rule ids the Schematron flagged that this implementation does not implement:
  BR-CL-04       on 1 document
  BR-CO-10       on 2 documents
  BR-CO-13       on 1 document
  BR-CO-14       on 1 document
  BR-CO-15       on 2 documents
  BR-CO-16       on 1 document
  BR-CO-17       on 2 documents
  BR-S-01        on 1 document
  BR-S-08        on 2 documents
  BR-S-09        on 2 documents

Two numbers, two meanings. "Correct as far as it goes" and "goes this far" are different claims and a report should not blur them.

3. A corpus where everything passes proves nothing

I pointed the harness at the official EN 16931 example documents and got zero failures on every one. Which is correct - they are the reference examples, they are supposed to pass - and completely useless. Two implementations agree trivially when there is nothing to disagree about.

You need documents that break. So generate them: take one known-good invoice and produce a variant per rule, each with exactly one thing wrong.

'br-co-17-vat-scaled-by-100' => [
    'rule' => 'BR-CO-17',
    'mutate' => function (DOMDocument $d) {
        foreach (xpath($d)->query('//cac:TaxSubtotal/cbc:TaxAmount') as $node) {
            $node->textContent = number_format(((float) $node->textContent) * 100, 2, '.', '');
        }
    },
],

Eleven fixtures, each reproducing a single failure. Checked in so they can be reviewed, generated by a script so they can be regenerated when the ruleset moves.

This is also the honest way to handle the reference documents your tax authority publishes. Croatia's twenty examples were published in December 2025; the rules were revised in March 2026 and nobody refreshed the examples. None of the twenty passes the current rules. So I never assert that they validate - I assert the measured outcome per file per rule id, and treat examples and rules as what they are: separate artifacts on separate release cycles.

What agreement does not mean

Here is the part I would want to read before trusting any of this.

Agreeing with the Schematron is not the same as being correct against the standard.

I had this backwards when I first published, and the correction is more interesting than the mistake. BR-CO-25 is not missing from Schematron 1.3.16 by oversight. It was removed there, and it is present in 1.3.13, 1.3.14 and 1.3.15 - asserted twice in 1.3.15's compiled UBL XSLT, and in the CII binding too:

((. > 0) and (exists(//cbc:DueDate) or exists(//cac:PaymentTerms/cbc:Note))) or (. <= 0)

So a positive amount due for payment with no due date and no payment terms was fatal under 1.3.15 and is valid under 1.3.16. Anyone who implemented that rule from the 2017 standard text is now rejecting compliant invoices.

Which flips the conclusion. Diffing against a pinned artifact is not a weaker substitute for checking against the standard's text - it is the thing you actually want, because the artifact moves and the text does not.

BR-CO-27 turned out to be a variation on the same theme rather than a counter-example, and I got this wrong before checking properly. It is absent from every UBL binding from 1.3.13 to 1.3.16, so a UBL-only implementation never had it to lose. But it is present in the CII binding for 1.3.13 and 1.3.14 and gone from 1.3.15 onward. Same story as BR-CO-25, one binding over and one release earlier.

Which is worth saying plainly: "missing from the Schematron" is not one fact. It depends on which binding and which release you mean, and a rule can be live in CII while it has never existed in UBL. If your implementation is bound to one syntax, that is the only history that applies to you.

The pin needs a second axis, then. A checksum tells you the ruleset file changed; it does not tell you whether anything you care about changed with it. So a ruleset can now pin several versions, and the corpus records which documents get a different verdict from each:

schematron-diff flips en16931-ubl 1.3.15 1.3.16
# 1 document(s) change verdict between 1.3.15 and 1.3.16:
#   br-co-25-payable-without-due-date.xml
#     1.3.15     BR-CO-25
#     1.3.16     (clean)
#
#   stopped firing in 1.3.16: BR-CO-25

A version bump becomes a decision you make rather than a behaviour change you discover. Thanks to Tobias for both the correction and the fixture idea.

So the claim this tool supports is narrow and worth stating precisely: my transcription matches the artifact I transcribed it from. That is a real claim and nobody had checked mine before. It is not my validator makes your invoice valid, and no amount of green CI turns one into the other.

I now report coverage per layer rather than as one number, for the same reason. "62 of 62" was true of the Croatian overlay and read as "fully validated", when three layers sat underneath it that I do not implement at all.

Run it on yours

I pulled the harness out into its own package, because the one part of this that is not Croatian was buried inside a Croatian package:

composer require --dev stboris/schematron-diff
vendor/bin/schematron-diff run en16931-ubl

It ships with the EN 16931 UBL Schematron and a corpus of deliberately broken documents, so it does something the moment you install it. Point it at your own ruleset by declaring one in schematron-diff.json and writing a thin adapter around your validator:

{
    "rulesets": {
        "my-cius": {
            "schematron": "resources/rules.sch",
            "corpus": "tests/corpus",
            "implementation": "App\\Testing\\MyValidatorAdapter",
            "scope": ["XX-BR-"],
            "sha256": "..."
        }
    }
}

The corpus is usable without the tool and without PHP: each one ships an expected.json listing the rule ids every document should produce, measured by running the Schematron rather than transcribed from the spec. Take the XML and that manifest into a .NET, Java or Python suite and run them with whatever you already have.

github.com/stboris/schematron-diff - MIT. A local JVM is used if you have one, otherwise a throwaway container; neither becomes a dependency of the code being tested.

If you run it for Germany, Poland, Italy or France, I would genuinely like to hear what you find - especially if it is a category of drift I have not hit yet.

And if you are about to reimplement a Schematron by hand: build this first. I built it second, and it immediately told me about a bug that had been shipping.