Cost-effective End-to-end Information Extraction for Semi-structured Document Images

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3375–3383 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3375 Cost-effective End-to-end Information Extraction for Semi-structured Document Images Wonseok Hwanga,∗ Hyunji Leeb,∗ Jinyeong Yimc Geewook Kimc Minjoon Seob,∗ aLBox bKAIST cClova, NAVER wonseok.hwang@lbox.kr hyunji.amy.lee@kaist.ac.kr {jinyeong.yim,gw.kim}@navercorp.com minjoon@kaist.ac.kr Abstract A real-world information extraction (IE) system for semi-structured document images often involves a long pipeline of multiple modules, whose complexity dramatically increases its development and maintenance cost. One can instead consider an end- to-end model that directly maps the input to the target output and simplify the entire process. However, such generation approach is known to lead to unstable performance if not designed carefully. Here we present our recent effort on transitioning from our existing pipeline-based IE system to an end-to-end system focusing on practical challenges that are associated with replacing and deploying the system in real, large-scale production. By carefully formulating document IE as a sequence generation task, we show that a single end-to-end IE system can be built and still achieve competent performance. 1 Introduction Information extraction (IE) for semi-structured documents is an important first step towards automated document processing. One way of building such IE system is to develop multiple separate modules specialized in each sub-task. For example, our currently-deployed IE system for name card and receipt images, POT (Hwang et al., 2019),1 consists of three manually engineered modules and one data-driven module (Fig. 1a). This system first accepts text segments and their 2D coordinates (we dub it “2D text”) from an OCR system and generates pseudo-1D-text using a serializer. The text is then IOB-tagged and mapped to a raw, structured parse. Finally, the raw parse is normalized by trimming and reformatting with regular expressions. ∗ Most work done while these authors were at NAVER. 1In October 2020, the system receives approximately 350k name cards and 650k receipts queries per day. a b IE System doc. images parses { store name: "WYVERN STORE", store tel: "13-806-4852" } ..., { menu name: "Polyjuice portion", count: "2", unit price: "1,000", price: "2,000" }, { menu name: "Cheap wand", count: "1", ... OCR Tag2parse Serializer Tagging model Parse normalizer doc. images e2e Serializer Tagging model parses OCR GT tags Inference routine Training routine GT: Ground truth Loss func. Parse annotator Tag annotator Loss func. 2D text GT 2D text 2D text tagged text tagged text raw parse parse parse GT parse 1D text 1D text { store name: "WYVERN STORE", store tel: "13-806-4852" } ..., { menu name: "Polyjuice portion", count: "2", unit price: "1,000", price: "2,000" }, { menu name: "Cheap wand", count: "1", ... line number Figure 1: The scheme of (a) our tagging based IE system and (b) the end-to-end IE system proposed in this study. Although POT has shown satisfactory performance to its customers, its long pipeline increases the development cost: (1) each module should be engineered and fine-tuned by experts per domain, and (2) each text segment should be annotated manually for the training. As …

Content truncated to 500 words