Document extraction demos beautifully. You upload a clean PDF invoice, the fields come back correct, everyone is impressed.
Then production arrives, and production is a photograph of a document taken at an angle in bad light with a stamp across the total.
The pipeline, not the model call
Extraction is one stage in a sequence, and the other stages are where the reliability comes from.
Ingestion. Accept whatever arrives — PDF, image, scan, email attachment, office format — and normalise it. Deskew rotated pages, enhance contrast, split multi-document files.
OCR. For anything not already machine-readable. Quality here caps everything downstream: text the OCR missed cannot be extracted by any model.
Layout analysis. Understanding structure — tables, columns, headers, which text belongs to which section. Extraction without layout context confuses a line item with a total.
Extraction. The model call. Pull the fields you need.
Validation. Does the extracted data have the right shape? Do the line items sum to the total? Is the date plausible?
Routing. Confident results proceed. Uncertain ones go to a human with context attached.
Multi-pass beats single-pass
A single extraction call over a complex document underperforms consistently.
What works better: a first pass to classify the document and identify regions, then targeted extraction per region, then a reconciliation pass checking internal consistency.
More calls and more cost, and materially better results on documents with any structural variation. It also produces better confidence signals, because disagreement between passes is itself informative.
Confidence is worth more than accuracy
This is the point teams take longest to internalise.
A system that is right 94% of the time and knows which 6% it is unsure about is far more useful than one that is right 96% of the time with uniform confidence. The first can route uncertainty to a human. The second silently produces wrong answers that look identical to right ones.
So calibrate confidence and make it act: threshold for automatic acceptance, threshold for review, threshold for rejection. Tune those against your own error tolerance, which differs enormously between a receipt and a contract.
Build the review console properly
Whoever works the review queue will spend their day in it, and its quality determines your throughput more than the model does.
What it needs: the document and the extraction side by side, the uncertain fields highlighted, the ability to correct in place rather than retype everything, and keyboard-driven navigation because volume review is a keyboard task.
Corrections should feed back into the golden dataset. That is how the system improves — every human correction is a labelled example you did not have to create.
Measure against your documents
Generic benchmarks tell you almost nothing about performance on your specific document types.
Build a golden dataset from your own documents, with correct answers, covering the range you actually see including the awkward ones. Run every change against it.
Without this you are changing prompts and hoping, and you will not notice that the fix for one issuer’s layout broke another’s.
Handle the awkward reality
Real document sets contain: mixed languages, handwriting, stamps and signatures obscuring text, photographs rather than scans, multiple documents in one file, near-duplicates, and formats that changed because an issuer redesigned their template.
None of these are edge cases. They are the normal composition of a production queue, and a pipeline built without them will be rebuilt.
Where the data sits
Documents frequently contain sensitive information, which makes inference location an architecture decision.
On-premise or in-region processing where data cannot leave, zero-retention arrangements otherwise, and explicit decisions about what gets logged — because logs of a document pipeline contain the documents.
Your team sets the requirements. Building for them from the start is far cheaper than relocating a pipeline later.
Common questions
How does AI document processing work?
A pipeline rather than a single call: ingestion and format normalisation, OCR for anything not already text, layout analysis to understand structure, extraction of the fields you need, validation against expected shapes, and a review path for anything low-confidence. The model call is one stage. The stages around it are what make it work on documents that were photographed at an angle.
Why does document AI fail on real documents?
Because real documents are nothing like test documents. They are scanned at angles, partially handwritten, in mixed languages, with stamps over text and inconsistent layouts between issuers. A pipeline built against clean PDFs collapses on the first batch of phone photographs, which is what production actually contains.
How accurate is AI document extraction?
It depends entirely on the document type and quality, which is why the useful approach is to measure against your own documents rather than quote a benchmark. What matters more than headline accuracy is calibrated confidence - a system that knows when it is uncertain and routes those cases to a human is far more valuable than one that is slightly more accurate and uniformly confident.
Should AI document processing have human review?
Yes, for anything consequential, routed by confidence rather than reviewing everything. The design that works is a pipeline that handles the clear majority automatically and escalates the uncertain remainder with the relevant context attached, so a human decides quickly rather than re-reading the whole document.
Does Corum8 build document processing systems?
Yes. We build ingestion and OCR pipelines, extraction with multi-pass reasoning, validation, confidence routing and the review console your team works in, measured against your own golden dataset rather than a generic benchmark.
