2000 identifications per second: how to select the right table on a document without neural networks
Smart Engines
Meta
Smart Engines has developed a method to identify a target table among multiple table-like regions on a document using regular expressions instead of neural networks. Each table region is converted into a one-dimensional string encoding its structure and text content, which is then matched against regex patterns. The approach runs about 2000 identifications per second on a mobile processor and improves recognition quality on real documents.
Smart Engines has developed a method for identifying the correct table among multiple table-like regions on a document without using neural networks. The system first detects potential table regions, then converts each region into a one-dimensional string that encodes both its geometric structure and text content. This is done by projecting vertical lines and OCR words onto the X-axis, marking positions as line quality digits, text markers, or empty dots, and merging them into a compact signature. The identification task then reduces to checking this string against regular expressions that describe the expected table structure. Two scenarios are used: fixed structure identification for standardized forms like invoices, and characteristic column identification for tables where column order may vary. Tests on Russian documents (acts and invoices) showed reductions in row and column detection errors by up to 35.87% and 26.36% respectively, with cell recognition accuracy improving by up to 9.97%. The algorithm runs in 0.49 ms per region on ARMv8 64-bit, about 2000 checks per second, compared to 1-3 seconds per candidate using an LLM approach with Llama 3 on a PC. The method was also applied to classify document types based on table structure, achieving 100% precision and recall for five out of seven types, outperforming a baseline field-based classification.
- Abbreviations
- OCR = Optical Character Recognition — оптическое распознавание символов
- LLM = Large Language Model — большая языковая модель
- ARMv8 = Advanced RISC Machine version 8 — архитектура ARM версии 8
Source: Habr — хаб ML —
original
