Document Information Extraction via Global Tagging

Document Information Extraction via Global Tagging Shaojie He1,2, Tianshu Wang2, Yaojie Lu2, Hongyu Lin2∗, Xianpei Han2∗, Yingfei Sun1, Le Sun2 1University of Chinese Academy of Sciences, Beijing, China 2Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China {heshaojie2020,tianshu2020,luyaojie,hongyu,xianpei}@iscas.ac.cn yfsun@ucas.ac.cn, sunle@iscas.ac.cn Abstract Document Information Extraction (DIE) is a crucial task for extracting key information from visually-rich documents. The typical pipeline approach for this task involves Optical Charac- ter Recognition (OCR), serializer, Semantic Entity Recognition (SER), and Relation Extraction (RE) modules. However, this pipeline presents significant challenges in real-world scenarios due to issues such as unnatural text order and error propagation between different modules. To address these challenges, we propose a novel tagging-based method – Global TaggeR (GTR), which converts the original sequence labeling task into a token relation classification task. This approach globally links discontinuous semantic entities in complex layouts, and jointly extracts entities and relations from documents. In addition, we design a joint training loss and a joint decoding strategy for SER and RE tasks based on GTR. Our experiments on multiple datasets demonstrate that GTR not only mitigates the issue of text in the wrong order but also improves RE performance. 1 Introduction Document Information Extraction (DIE), which is to extract key information from document with com- plex layouts, has become increasingly important in recent years (Zhang et al., 2022; Hong et al., 2022). It not only enables us to efficiently compress document data, but also facilitates the retrieval of impor- tant information from documents. A typical pipeline approach for the DIE task is depicted in Figure 1(a) (Denk and Reisswig, 2019; Hwang et al., 2021a). First, the document with complex layout is con- verted into text blocks using Optical Character Recognition (OCR) tools. Next, the serializer module organizes these text blocks into a more appropriate order. Finally, the well-ordered text blocks are in- put sequentially into the Semantic Entity Recognition (SER) and Relation Extraction (RE) modules to extract key-value pairs. However, the pipeline approach in Figure 1(a) presents significant challenges in real-world scenarios. (1) Mainstream models for the DIE task, such as LayoutLM (Xu et al., 2020), LayoutLMv2 (Xu et al., 2021) and LayoutXLM (Xu et al., 2022), usually use sequence labeling in the Beginning-Inside–Outside (BIO) tagging schema, which assume that tokens belonging to the same semantic entity are grouped together after serialization. If the serializer module fails to order the text blocks correctly, the final performance can be severely impacted. A potential solution is to train a strong and robust serializer module, but this is difficult due to the labor-intensive labeling process under rich and diverse styles of documents; (2) In addition to the issue of text order, this pipeline also suffers from error propagation when using a SER module and a RE module. In research settings, the results of the SER and RE tasks are generally tested separately, with the ground truth of the SER results being used as default auxiliary information for the RE task. However, in real-world scenarios, the SER …

Content truncated to 500 words