Schema-Driven Extraction: Why a TemplateId Beats Hardcoded Regex in Your App

Every invoice-parsing codebase starts the same way. Someone writes a regular expression that finds the total, it works on the twelve sample invoices sitting in the test folder, and it ships. Then a vendor moves the total from the bottom right of page one to the top of page two, relabels it "Amount Payable" instead of "Total Due", and the pattern that worked for eighteen months returns null. Nothing crashed. Nothing threw. A number simply stopped arriving, and the first person to notice is whoever reconciles the ledger three weeks later.

The reflex at that point is to make the regex smarter. Add an alternation for the new label. Add a lookahead for the currency symbol. Add a fallback that scans the last page too. That is how a forty-character pattern becomes a four-hundred-character pattern that exactly one person on the team is willing to touch.

Here is the part worth arguing about. The problem was never that the regular expression was badly written. The problem is where it lives.

A pattern describes position, a schema describes meaning

A regex encodes a claim about where a string sits and what characters surround it. It says: after the literal text Total Due, skip some whitespace, expect a currency symbol, capture the digits. Every one of those is an assumption about layout, and layout is the least stable property of a business document. Vendors redesign templates. Accounting systems get replaced. A scanned copy shifts a label onto its own line. None of that changes what the number means, and all of it breaks the pattern.

A schema encodes the other thing entirely. It says: there is a field called TotalAmount, it is a number, and it represents the grand total including taxes. No position, no delimiter, no assumption about which page it lives on. That description is still true after the vendor's redesign, because it was never a statement about the document's geometry.

So the interesting question is not regex versus AI. It is where your extraction logic is allowed to live.

Three places extraction logic can live

Tier 1, in your application code. The regex is a string literal in your repo. Changing it means a pull request, a review, a build, and a deploy. Every consumer of that logic has to be on the same release to agree on what a total is.

Tier 2, in a parse template referenced by an id. The pattern still exists, but it lives in a named template you edit in a dashboard. Your code sends a TemplateId instead of a pattern. Fixing a vendor redesign is an edit, not a release.

Tier 3, in a document schema referenced by an id. No pattern at all. You describe the fields in natural language, and the engine locates them semantically. Your code sends an AnalyzerId.

PDF4me exposes tiers 2 and 3 as two separate, coexisting products, and the docs are unusually direct about the difference. Worth walking both.

Tier 2: Parse Document and the TemplateId

The Parse Document endpoint runs a saved parse template against a PDF and returns the extracted fields as JSON in one REST call. You build the template in the dashboard: upload a sample PDF, draw capture areas, configure keys with a Regex Expression for stable patterns or a JavaScript Expression for conditional rules. The full setup walkthrough, including working classifier samples, is in Prepare Parse Info for Document.

The endpoint is POST /api/v2/ParseDocument on the base URL https://api.pdf4me.com, documented in Connect to the PDF4me V2 API.

The minimum payload is three fields, which surprises people who expect the template to be mandatory:

{
  "docContent": "JVBERi0xLjQK...",
  "docName": "invoice.pdf",
  "async": true
}

That call still returns useful default fields such as documentType and pageCount, just nothing keyed to your own capture fields. To get those, reference the template:

{
  "docContent": "JVBERi0xLjQK...",
  "docName": "invoice.pdf",
  "TemplateId": "12345678-1234-1234-1234-123456789abc",
  "ParseId": "87654321-4321-4321-4321-cba987654321",
  "async": true
}

Three details in that payload are worth pinning down.

TemplateId is a GUID the dashboard assigns when you click Save Changes, and it stays stable for the life of the template. There is also a TemplateName lookup, and the docs are explicit that you should prefer the id in production, because renaming a template silently breaks every call that referenced it by name. That is the same class of bug as the vendor redesign, just moved one layer up.

ParseId is a GUID you generate per call, client side, with uuid.uuid4 in Python, Guid.NewGuid in C#, or UUID.randomUUID in Java. It is not validated against a registry. Its job is correlation in your own logs and audit trail.

async decides the response shape. With false you get HTTP 200 and the parsed JSON immediately. With true you get HTTP 202 and a Location header to poll.

The response is JSON, not a binary file, which is a real difference from the Protect, Compress, and Convert endpoints in the same API:

{
  "parsedData": {
    "invoiceNumber": "INV-2024-001",
    "invoiceDate": "15/01/2024",
    "totalAmount": "$1,250.50",
    "customerName": "Acme Corporation"
  },
  "documentType": "invoice",
  "pageCount": 1
}

Note that totalAmount comes back as the string "$1,250.50", currency symbol and thousands separator included. A regex-driven template captures what the document says, not a normalised number. Whatever you do about that, do it deliberately rather than discovering it when a downstream INSERT rejects the row.

The async path in Python

For anything over a few megabytes, or any batch, you want async: true and a polling loop. This is adapted from the official Parse Document Python sample in the MIT-licensed pdf4me-api-samples repo, trimmed to the parts that matter:

import base64
import json
import time
import requests

API_KEY = "your-api-key"
URL = "https://api.pdf4me.com/api/v2/ParseDocument"

with open("invoice.pdf", "rb") as f:
    doc_content = base64.b64encode(f.read()).decode("utf-8")

payload = {
    "docContent": doc_content,
    "docName": "invoice.pdf",
    "TemplateId": "12345678-1234-1234-1234-123456789abc",
    "ParseId": "87654321-4321-4321-4321-cba987654321",
    "async": True,
}

headers = {
    "Authorization": f"Basic {API_KEY}",
    "Content-Type": "application/json",
}

response = requests.post(URL, json=payload, headers=headers, timeout=300)

if response.status_code == 200:
    print(json.dumps(response.json(), indent=2))

elif response.status_code == 202:
    location = response.headers.get("Location")
    if not location:
        raise RuntimeError("202 returned with no Location header")

    for _ in range(15):
        time.sleep(10)
        poll = requests.get(location, headers=headers, timeout=60)
        if poll.status_code == 200:
            print(json.dumps(poll.json(), indent=2))
            break
        if poll.status_code != 202:
            raise RuntimeError(f"Poll failed: {poll.status_code} {poll.text}")
    else:
        raise TimeoutError("Parse did not complete within the retry budget")

else:
    raise RuntimeError(f"Request failed: {response.status_code} {response.text}")

Two honest notes on that sample. The published version passes verify=False to requests, which disables TLS certificate verification. Do not carry that into production code, it is there to smooth over local certificate stores and it is not something you want talking to a payments-adjacent endpoint. And the 15 retries at 10 seconds is a budget of two and a half minutes, which is a starting point, not a guarantee. Size it against your own documents and treat the exhausted loop as a real failure path rather than a silent pass.

Every language sample lives under the same tree if Python is not your stack: C#, Java, JavaScript, Salesforce, and n8n are all in the Parse Document sample folder.

Tier 3: the AnalyzerId and a document schema

Tier 2 moved the pattern out of your repo. It did not stop being a pattern. Tier 3 is where the argument in the title actually lands.

The AI Document Parser using Parse guide describes a different dashboard object: an Analyzer. You create one, pick the Parse type, give it an Analyzer Id, and instead of drawing capture areas on a sample PDF you paste a Document Schema in JSON. The docs state the distinction plainly: the older Parse Document setup applies Regex Expression or JavaScript Expression to drawn capture areas on a sample PDF, while the AI Analyzer reads the document semantically from natural-language field descriptions, and needs no sample PDF at setup time.

A schema has two top-level keys, description and fields:

{
  "description": "Extracting data from supplier invoices.",
  "fields": [
    {
      "fieldName": "InvoiceNumber",
      "fieldType": "string",
      "fieldDescription": "Unique invoice identifier, sometimes shown as Invoice No. or INV."
    },
    {
      "fieldName": "InvoiceDate",
      "fieldType": "date",
      "fieldDescription": "Date the invoice was issued."
    },
    {
      "fieldName": "DueDate",
      "fieldType": "date",
      "fieldDescription": "Date payment is due, sometimes shown as Payment Due or Net Due."
    },
    {
      "fieldName": "VendorName",
      "fieldType": "string",
      "fieldDescription": "Company name of the supplier or vendor sending the invoice."
    },
    {
      "fieldName": "TotalAmount",
      "fieldType": "number",
      "fieldDescription": "Grand total in the invoice currency, including taxes."
    }
  ]
}

Read the fieldDescription strings again. That is the whole shift. "sometimes shown as Invoice No. or INV." is the alternation you would have written as regex, expressed as a sentence instead. The docs are direct that this string is what the engine leans on to locate each value, and that including alternate labels and example formats is the work. Vague descriptions are the failure mode here, the same way a brittle anchor was the failure mode before.

The field attributes, exactly as documented:

fieldMethod: "generate" is the answer to the "$1,250.50" problem from tier 2, and fieldType: "table" is the answer to line items. A table field carries its own nested fields array, one entry per column:

{
  "fieldName": "lineItems",
  "fieldType": "table",
  "fieldDescription": "All product / service rows from the invoice table. Be careful, sometimes a row can be part of the next item like when description goes over one line, but it's of a single item.",
  "fields": [
    {
      "fieldName": "itemNumber",
      "fieldType": "string",
      "fieldDescription": "Product number, product id number or product code"
    },
    {
      "fieldName": "hsnCode",
      "fieldType": "string",
      "fieldDescription": "HSN / SAC code (4 to 8 digit)"
    }
  ]
}

That fieldDescription on lineItems is worth staring at. It handles a wrapped description line spilling into what looks like a second row. Try expressing that constraint as a regular expression and see how far you get.

Calling the saved Analyzer takes four fields, and notably no template id and no parse id:

{
  "docName": "purchase_order.pdf",
  "docContent": "BASE64_ENCODED_PDF_CONTENT",
  "AnalyzerId": "purchase_order_parser",
  "async": false
}

AnalyzerId is just the string you typed when creating the Analyzer, not a generated GUID, and it does not change when you edit the schema. That makes it stable, and it also makes it your problem: the docs recommend versioning it (invoice_parser_v1, invoice_parser_v2) when you make a breaking schema change, so live automations can migrate on their own schedule. Treat the Analyzer Id like a public API surface, because that is what it is.

Build these at the AI Document Parser dashboard, with an API key from the same place.

Mixed batches break patterns before layout does

There is a failure mode that regex handles even worse than a redesign, and it is the ordinary case for anyone processing an inbox. The batch is mixed. Invoices, receipts, contracts, bank statements, and the occasional scanned ID all arrive through the same folder, and a pattern written for one document type produces confident nonsense when handed another. A total-matching regex will happily find a number on a contract.

The documented answer is to identify the document before extracting from it. Classify Document takes docContent and docName, with async optional, and no template or schema at all:

{
  "docContent": "JVBERi0xLjQK...",
  "docName": "invoice.pdf",
  "async": true
}

The documented response shape:

{
  "documentType": "invoice",
  "category": "financial",
  "confidence": 0.95,
  "metadata": {
    "pageCount": 1,
    "createdDate": "2024-01-15T10:30:00Z"
  }
}

That is the shape from the docs, not a benchmark. What matters architecturally is that confidence exists at all, because it gives you a third branch. Invoices route to the invoice schema, bank statements to the statement schema, and anything under your own threshold routes to a human queue instead of into the ledger. Set that threshold from your own measured results on your own documents.

Classify then parse is a two-call pipeline a regex cannot express, because a pattern has no way to tell you it was pointed at the wrong kind of document. It only tells you whether it matched. Configuring the classifier with your own document types is covered in Set Up Classify Document, and the AI Analyzer equivalent, where one Analyzer routes between document variants, is in AI Document Parser using Classify.

Try it before you write the integration

Both endpoints have an interactive tester where you can upload a real file and watch the actual response come back: Parse Document API Tester and Classify Document API Tester. The API Tester overview covers how it works.

This is not a throwaway suggestion. While writing this piece I found that PDF4me's own surfaces describe Parse Document in more than one way depending on which page you land on, because the classic template product and the newer AI Analyzer product overlap in naming. Confirm the exact request and response shape for your own integration against a live call rather than any parameter table, this one included.

The same idea outside a codebase

None of this requires raw REST. Parse Document is a native step in the major automation platforms: Parse a Document in Make, Parse Document in Power Automate, Parse Document in Zapier, and Parse Document in n8n.

The AI Analyzer has its own dedicated actions, and the same Analyzer Id works in all of them without recreating the schema per platform: AI Document Parser in Make, in Power Automate, in Zapier, and in n8n.

That portability is the real payoff of putting the logic behind an id. The schema is defined once, and the REST service, the Make scenario, and the n8n workflow all agree on what a total is, because they are all reading the same definition.

When the regex was right all along

The contrarian case has a limit, and pretending otherwise would be dishonest.

Schema-driven extraction is the wrong tool when the thing you are looking for genuinely is a fixed pattern in a document you control. An internal report your own system generates, with a reference code that always matches the same format in the same place, does not need a model to interpret it. For that, Extract Text by Expression runs a single regex against a PDF with no saved template at all, and is a better fit than either tier above.

The same applies to structural extraction rather than semantic extraction. If the document is a filled AcroForm, its values are already named and structured, so Extract Form Data from PDF reads them directly. If what you need is a whole table as rows and columns rather than a handful of named fields, Extract Table from PDF is the endpoint for that.

One more limit worth naming: scanned documents. The AI Analyzer works on a text layer, so a scan has to go through OCR before any of this applies.

The dividing line is straightforward. Use a pattern when you own the layout and it is not going to move. Use a schema when someone else owns the layout, which describes almost every document that arrives from outside your company.

And whichever you pick, keep it out of your deploy pipeline. The reason a TemplateId beats a hardcoded regex has less to do with regex being bad and more to do with the fact that a vendor changing their invoice template should never require you to cut a release.

Website: pdf4me.com
Documentation: docs.pdf4me.com
Developer portal: dev.pdf4me.com