Reading is easy. Getting it right is engineering.

Accuracy you can inspect.

An invoice total off by one digit can look perfectly plausible. A bank statement can have accurate amounts and still be missing a transaction. In accounting, these distinctions matter.

We evaluate extraction field by field, inspect failures, and test changes against reference answers. This report presents Ogment’s updated production results and a historical example of how we improve the system.

Tax amount accuracy · Ogment

99.83%

Total accuracy · SROIE receipts

99.90%

Bank statement files completed

99.75%

Evaluation sample: For the updated evaluation, blurry or dirty documents were replaced with other documents. The results describe this replacement sample, not the full range of incoming document quality. The bank-statement sample contains 400 files. These results are separate from the original September benchmark.

These are extraction results, before accounting judgment or human correction. A field score is not the percentage of entirely correct documents. The benchmark does not measure Swiss VAT treatment, account assignment, or end-to-end bookkeeping accuracy.

The document is the evidence.

A structured value should be easy to trace back to its source. An arithmetic match supports review; it does not prove tax treatment or every field correct.

• Invoices

  Subtotal plus printed tax should equal the total. Each extracted value still needs to agree with the document.

• Difficult receipts

  When a tax line is unclear, an inferred amount is not printed evidence. A missing value and a confidently wrong value create different work for the reviewer.

• Bank statements

  Opening balance plus signed transactions should equal the closing balance. A missing or reversed row can break this check even when other amounts were read accurately.

There is no single “accuracy.”

Document fields and matching scores are different tasks. We report them separately so a strong result on one cannot hide a weakness on another. The figures below show the updated production results; they are not a rerun of the earlier four-setup comparison.

Clean invoices are only half the story.

Real receipts bring unfamiliar layouts, compact dates, and ambiguous tax lines. Looking at individual datasets makes the remaining errors visible, even after document-quality screening.

• Read the printed tax. Don’t infer the treatment.

  Extracting the tax amount printed on a receipt is separate from deciding whether it is deductible or which VAT code applies. This benchmark tests the former.

• Make the difficult cases visible.

  For a reviewer, a missing value and a confidently wrong value create different work. Future evaluations should measure both, plus which mistakes the system flags.

A failure becomes a test.

One earlier version often classified receipts as invoices. We revised the classification instructions and reran the same document set. The effect was large on one receipt dataset, and a small regression remained elsewhere.

1. Inspect the failure Was the source unreadable, the instruction ambiguous, or the extracted structure incomplete?
2. Make the rule explicit Use the failure to clarify expected behavior, including how to represent missing information.
3. Check every slice Measure improvements and regressions across document types, then confirm on unseen documents.

Every row has to be found.

An accurate amount on a matched transaction is useful. It does not tell you whether the entire statement was extracted. We report file completion separately.

Matched-transaction accuracy excludes rows that could not be paired with the reference. A complete evaluation also needs missed-row and extra-row measurements.

Accuracy and review belong together.

The next question is practical: how accurate are the documents that pass validation, and how many require review? Field-level results alone do not answer it.

The proposed metric is whether every required critical field is correct on accepted documents, before human correction. It needs independently labeled documents and thresholds based on validated system signals. No calibrated document-acceptance result is claimed here.

The denominator matters.

A high score is meaningful only when the task, test set, and scoring rules are clear.

• Reference answers

  Public datasets provide labeled document fields. Missing outputs count as errors when a reference answer exists. Fields without reference labels are excluded from that field’s score.

• Exact match

  Dates are normalized to the same date representation. Amounts must match to the cent. Document numbers are normalized for spaces, punctuation, and case.

• Evaluation sample

  For the updated evaluation, blurry or dirty documents were replaced with other documents. The results describe this replacement sample, not the full range of incoming document quality. The bank-statement sample contains 400 files. These results are separate from the original September benchmark.

• Comparisons need the same protocol

  A replacement sample changes what a score describes. Updated production scores cannot establish an improvement over the original sample or rank against models measured only on the earlier benchmark.

• Anthropic · Sonnet 4.6 evaluation presentation
• OpenAI · GPT-5 evaluations and reliability

Updated figures are Ogment’s reported evaluation results. The historical classification values come from the September report. The research links illustrate evaluation presentation; they are not sources for Ogment’s scores.