Lightweight Spatial Modeling for Combinatorial Information Extraction From Documents

Findings of the Association for Computational Linguistics: EACL 2023, pages 1471–1484 May 2-6, 2023 ©2023 Association for Computational Linguistics Lightweight Spatial Modeling for Combinatorial Information Extraction From Documents Yanfei Dong12, Lambert Deng1, Jiazheng Zhang1 Xiaodong Yu1, Ting Lin1, Francesco Gelli1, Soujanya Poria DeCLaRe, Wee Sun Lee2 1 PayPal 2 National University of Singapore DeCLaReDeCLaRe Lab, Singapore University of Technology and Design {dyanfei, yuadeng, jiazzhang, xiaodyu, tinlin, fgelli}@paypal.com sporia@sutd.edu.sg, leews@comp.nus.edu.sg Abstract Documents that consist of diverse templates and exhibit complex spatial structures pose a challenge for document entity classification. We propose KNN-Former, which incorporates a new kind of spatial bias in attention calculation based on the K-nearest-neighbor (KNN) graph of document entities. We limit entities’ atten- tion only to their local radius defined by the KNN graph. We also use combinatorial match- ing to address the one-to-one mapping prop- erty that exists in many documents, where one field has only one corresponding entity. More- over, our method is highly parameter-efficient compared to existing approaches in terms of the number of trainable parameters. Despite this, experiments across various datasets show our method outperforms baselines in most en- tity types. Many real-world documents exhibit combinatorial properties which can be lever- aged as inductive biases to improve extraction accuracy, but existing datasets do not cover these documents. To facilitate future research into these types of documents, we release a new ID document dataset that covers diverse tem- plates and languages. We also release enhanced annotations for an existing dataset.1 1 Introduction Structured document information extraction (IE) attracts increasing research interest due to the surg- ing demand for automatic document processing, with practical applications in receipt digitization, workflow automation, and identity verification etc. Recent state-of-the-art methods for processing documents with complex layouts extensively ex- ploit layout information, such as position, relative distance, and angle, with transformer-based mod- els. Spatial modelling is a key contributing factor to the success of these methods ( Xu et al. 2020, Appalaraju et al. 2021, Xu et al. 2021, Hwang et al. 2021). However, absolute coordinates, pair-wise 1https://github.com/miafei/knn-former relative Euclidean distance, and angle are insuffi- cient to capture the spatial relationship in complex layouts. Two document entity pairs could carry different importance despite having the same posi- tion and distance, due to the presence or absence of other entities positioned between the pairs. We believe that spatial information can be better ex- ploited for document entity classification. We propose KNN-Former, a parameter-efficient transformer-based model that extracts informa- tion from structured documents with combinato- rial properties. In addition to relative Euclidean distance and angle embeddings as inductive bi- ases (Hwang et al., 2021), we introduce a new form of spatial inductive bias based on the K-Nearest Neighbour (KNN) graph which is constructed from the document entities and integrate it directly into the attention mechanism. Specifically, we first con- struct a KNN graph based on the relative Euclidean distance of document entities. Then we incorporate hop distance between entities, which is defined as the shortest path between two entities on the KNN …

Content truncated to 500 words