Intelligent document processing (IDP) with low-code tools

OCR (Optical Character Recognition) converts images of text into machine-readable text characters — it turns a scanned image into a string of characters. IDP (Intelligent Document Processing) goes much further: it understands the document's structure and layout, classifies the document type, extracts specific fields with semantic meaning (identifying that "INV-2024-001" is an invoice number, not just a text string), validates extracted data against business rules, handles document variation (different invoice layouts from different vendors), and learns from corrections. IDP uses OCR as a first step but adds AI layers on top for understanding, extraction, and validation.

IDP platforms can process virtually any document type that contains extractable information: financial documents (invoices, purchase orders, receipts, bank statements, remittance advice); identity documents (passports, driving licences, national ID cards); legal documents (contracts, NDAs, lease agreements); insurance documents (claims forms, policy documents, medical records); HR documents (CVs, employment contracts, expense reports); tax forms (W2, 1099, VAT returns); logistics documents (bills of lading, customs declarations, delivery notes); and healthcare forms (referral letters, lab results, prescription forms). Pre-built models exist for common types; custom model training handles domain-specific document formats.

IDP accuracy on well-supported document types (printed invoices from major vendors, standard forms) typically achieves 95–99% field-level accuracy with modern platforms. Factors that reduce accuracy include: poor scan quality (low resolution, skew, shadows); handwritten content (much harder than printed text); unusual or complex document layouts not represented in training data; documents in languages the model was not trained on; and multi-column or multi-page table extraction. Most platforms report confidence scores per extracted field, allowing low-confidence extractions to be routed to human review. A well-configured IDP system with human-in-the-loop for exceptions typically achieves effective end-to-end accuracy of 99%+ even when raw model accuracy is lower.

LayoutLM (and its successors LayoutLMv2, LayoutLMv3) is a document understanding model from Microsoft Research that extends BERT to incorporate the 2D spatial layout of text in documents alongside the text content itself. Standard NLP models process text sequentially without knowing where on the page text appears. LayoutLM encodes both the text tokens and their bounding box coordinates (position on the page), enabling the model to understand that "Total" in the bottom right of an invoice refers to the grand total amount, not an item description — because it understands spatial relationships. LayoutLM-based models significantly outperform text-only models for document information extraction, particularly for complex multi-column layouts.

IDP integrates with ERP systems (SAP, Oracle, Dynamics, NetSuite) primarily via API — the IDP platform extracts structured data from documents and POSTs it to the ERP's API endpoints for invoice creation, purchase order matching, or master data lookup. Integration middleware (MuleSoft, Dell Boomi, Azure Logic Apps) often mediates between IDP output and ERP API formats, handling data transformation, error routing, and retry logic. For SAP specifically, IDP can integrate via SAP BTP Integration Suite, SAP Document Information Extraction (a native SAP IDP service), or direct RFC/BAPI calls from the IDP platform's integration connectors. The integration layer also retrieves master data (vendor lists, GL account codes) for validation during the IDP processing pipeline.

Straight-through processing means a document is processed from ingestion to output without any human intervention — the IDP system classifies, extracts, validates, and exports the data fully automatically. STP rate (the percentage of documents processed without human review) is a key IDP performance metric. A mature invoice processing deployment typically achieves 70–90% STP rate — meaning 70–90% of invoices are processed end-to-end automatically, with only 10–30% requiring human review for exceptions. Factors affecting STP rate: document quality, document type consistency, model accuracy, and the strictness of validation rules. Increasing STP rate is an ongoing optimisation process involving model retraining, rule refinement, and vendor engagement to standardise document formats.

Generative AI has significantly changed IDP by enabling zero-shot and few-shot document extraction — extracting data from new document types without training a custom model. You provide the LLM with a document (as an image or text) and a prompt specifying what fields to extract, and the model returns structured output. This reduces the time to deploy IDP for a new document type from weeks (training data collection, model training, validation) to hours or days. Multimodal LLMs (GPT-4 Vision, Claude, Gemini) can process document images directly, handling layouts that traditional OCR+NLP pipelines struggle with. The trade-offs are higher cost per page and less predictable output structure at very high volumes compared to purpose-trained extraction models.

IDP ROI is primarily driven by labour savings from eliminated manual data entry, supplemented by error reduction (cost of fixing data entry errors in downstream systems), faster processing cycle time (invoice processing from receipt to approval), and improved compliance (audit trail, consistent application of business rules). Typical benchmarks: accounts payable automation achieving STP of 80% on 1,000 invoices per month saves approximately 200 hours/month of manual processing time. At €25/hour fully loaded cost, this is €5,000/month or €60,000/year. IDP platform costs for this volume typically run €1,000–3,000/month, giving a 2–5× ROI. Larger volumes improve the economics further. Payback periods of 6–18 months are typical for AP automation IDP implementations.