The OCR that refuses to guess

The scanner produces a PDF, the model returns JSON, the report says 99.1% accuracy. That number reads like the end of the conversation, and it is the beginning: it does not say which fields were right, what happened to the other 0.9%, or whether the system, when it did not know, left the field empty or wrote its best guess anyway. That distinction decides whether invoice capture is a saving or a quiet error factory that surfaces at audit time.

The average hides exactly what costs money

A supplier invoice is not one data point. It is a header of a dozen fields of very different weight — tax ID, invoice number, supply date, payment reference, taxable base per rate, VAT amount, gross total — and a body of line items, each carrying an item code, quantity, unit of measure, price and discount. Collapse that into one percentage and the line items take over the denominator: a ten-line invoice has some fifty cells in the body against a dozen in the header.

A model that reads all fifty cells perfectly but misses the tax ID and the VAT amount still reports about 97% — and that invoice is unusable, because it posts to the wrong counterparty and lands in the wrong box of the VAT ledger. The average rewards the easy work: names and units are numerous and cheap to get wrong, while a handwritten payment reference or the base on a two-rate invoice is rare and expensive.

What a report worth reading looks like

Instead of one figure, each field type is measured on three outcomes: correct, refused (the system was not confident), and silently wrong. Only the third column is a defect. The first is work done, the second work handed over cleanly — a queue, not a failure.

Refuses-to-guess: the metric that measures honesty

The refuses-to-guess rate is the share of fields the system deliberately leaves empty and routes to a human instead of writing its most likely candidate. It sounds like an admission of weakness; it is one, and that is the point. Models do not do this on their own — there is always a top candidate, and „I don't know“ does not exist until someone builds it. Building it means a threshold per field, calibrated against a golden set. A threshold fine for a supplier name is catastrophic for a taxable base.

• Thresholds are per field: one threshold per document means the most expensive field shares the fate of the cheapest.
• Calibration runs on real documents — bad scans, faxed copies, two-rate invoices — not on intuition.
• Refusal must be cheap: focus lands on the disputed field, a crop of the original beside it, confirmation in one gesture.
• If refusals fall while silent errors rise, the threshold has slipped — a regression, not progress.
• A refused field blocks automatic posting; a warning next to an enabled Post button is not a control.

The price of a confident error

The asymmetry is measurable. An empty field costs an accountant thirty seconds. A confidently wrong field costs a chain that keeps running — precisely because nothing in it breaks.

1. 1

   The invoice arrives

   The model reads the standard-rate base as 1,180,000 instead of 1,080,000 — one digit, clean scan, high confidence. Gross total and VAT read correctly.

2. 2

   Posting succeeds

   VAT posted as read rather than derived from the base still balances. A control checking only debits against credits reports nothing.

3. 3

   The VAT ledger inherits the error

   The base lands in the wrong box. The base-to-VAT ratio is off, but that is checked on totals, not per document — and there it dissolves.

4. 4

   The return is filed, then discovery comes from outside

   The deadline passes and the period closes. Forty days later a supplier statement does not reconcile, or an auditor asks for the ledger document by document.

5. 5

   Remediation costs more than everything saved

   An amended return, a reversal and a repost in a closed period under the rules that applied then — and, worst, lost trust in every invoice the model read that month. The team rechecks everything by hand, and the automation is pointless while still on.

Duplicates are a first-class problem

In practice the same invoice arrives two or three times: as a PDF by email, then through SEF, Serbia's national e-invoicing platform, then as a scan attached to warehouse paperwork. The supplier sends a corrected version under the same number; a salesperson forwards a message accounting already processed. None of those paths knows about the others, so duplicate detection cannot be one comparison — signals differ in strength and reliability.

• The triple (tax ID, invoice number, year) is a hard key — but the invoice number is, per the table above, among the fields with the most OCR errors. A key is only as strong as the field behind it.
• Same supplier, amount and date, different number: a soft signal, usually a misread number, occasionally two genuine deliveries in one day.
• A content hash is strong for digital invoices and worthless for scans — two scans of one sheet never hash alike.
• The SEF UUID is the strongest signal there is, but only the document that came through SEF carries it — its PDF copy in an inbox does not.
• Reversals and credit notes resemble duplicates on every similarity measure and are not. Detection that cannot tell them apart blocks valid documents and gets switched off within a week.

This is where the two topics meet. If the invoice number was guessed, the duplicate key is wrong and the check passes quietly — it does not report an error, it reports that all is well. A guess in one field disables a control in another.

Receipt matching is where OCR stops being OCR

OCR that fills the header and stops is a typing robot. Value starts when the captured document finds its place in the chain: purchase order → goods receipt → invoice. That is 3-way matching, and the reason a supplier invoice is worth processing automatically at all.

1. Narrow the candidates: receipts from the same supplier, around the supply date, not yet invoiced.
2. If the invoice carries a purchase order number, match on it — the cheapest and most reliable link, and the reason to ask suppliers for it.
3. If not, match on lines: item code from the supplier catalogue mapping, quantity after unit-of-measure conversion, price from the contract.
4. Apply tolerances that are contracted, not invented: price variance, quantity variance, permitted rounding.
5. For imports, separate what capitalises into inventory from what does not — customs and freight belong in landed cost and arrive on a different invoice.
6. Propose the link with its reasoning — which receipt, on what criterion, with what variance — and wait for confirmation.

How NG One does this

NG One runs invoice capture as a human-in-the-loop flow rather than an autopilot: PDF or scan → extraction → supplier identification → duplicate detection → posting proposal → proposed link to the goods receipt → uncertain fields marked → human confirmation. The order matters: duplicates are checked before the posting proposal, because proposing a posting for an already posted document is wasted work at best. Alongside it runs an evaluation framework that is mandatory, not desirable — without it, everything above is an intention rather than a control.

• A golden set of documents and AI regression tests in CI: a model or prompt that drops accuracy on any field does not merge, like a failing unit test.
• Per-field metrics as the threshold, plus the refusal rate — measured field by field, never as one aggregate.
• An AI audit log with model and prompt version for every read: when an error surfaces six months later, it must be answerable which model read what, and who confirmed it.
• Prompt-injection tests from attached documents — an invoice is untrusted input, and text inside it is data, never instruction.
• Tenant isolation tests: a model processing one client's document sees nothing outside that client's permissions.

Six questions for any vendor

1. Show accuracy per field type, not the average — taxable base per rate, invoice number, payment reference.
2. What share of fields does the system refuse to guess, and how is the threshold tuned per field?
3. How many silent errors are on your golden set, and how large is it?
4. How is a duplicate detected when the invoice number was misread?
5. What happens when an invoice matches no goods receipt — propose, block, or post unmatched?
6. Is there a trail: which model, which prompt version, which field was uncertain, who confirmed it?

The first question usually gets a single percentage. The second usually goes unanswered — the metric does not exist, and a missing metric means the system is guessing and nobody is counting.