Representation Learning for Information Extraction from Form-like Documents

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6495–6504 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6495 Representation Learning for Information Extraction from Form-like Documents Bodhisattwa Prasad Majumder†♣ Navneet Potti♠ Sandeep Tata♠ James B. Wendt♠ Qi Zhao♠ Marc Najork♠ ♣Department of Computer Science and Engineering, UC San Diego bmajumde@eng.ucsd.edu ♠Google Research, Mountain View {navsan, tata, jwendt, zhaqi, najork}@google.com Abstract We propose a novel approach using represen- tation learning for tackling the problem of ex- tracting structured information from form-like document images. We propose an extraction system that uses knowledge of the types of the target fields to generate extraction candidates, and a neural network architecture that learns a dense representation of each candidate based on neighboring words in the document. These learned representations are not only useful in solving the extraction task for unseen docu- ment templates from two different domains, but are also interpretable, as we show using loss cases. 1 Introduction In this paper, we present a novel approach to the task of extracting structured information from form- like documents using a learned representation of an extraction candidate. Form-like documents like invoices, purchase orders, tax forms and insurance quotes are common in day-to-day business work- flows, but current techniques for processing them largely still employ either manual effort or brit- tle and error-prone heuristics for extraction. The research question motivating our work is the fol- lowing: given a target set of fields for a particular domain – e.g., due date and total amount for in- voices – along with a small set of manually-labeled examples, can we learn to extract these fields from unseen documents? Take, for instance, the domain of invoices, a doc- ument type that large enterprises often receive and process thousands of times every week (iPayables, 2016). Invoices from different vendors often present the same types of information but with dif- ferent layouts and positioning. Figure 1 shows the headers of invoices from a few different vendors †Work done during an internship at Google Research Figure 1: Excerpts from sample invoices from different vendors. Instances of the invoice_date field are highlighted in green. showing the invoice date (highlighted in green) and number in different layouts. Furthermore, in- voices from the same supplier even share similar presentation and differ only in specific values. We refer to this unit of visual pattern that is similar across a collection of documents as a template, and the fields of information that are common across templates in a domain as the schema. The schema consists of fields like invoice_date and total_amount, each associated with a type like date and currency. Extracting values for these fields from a given document, particularly one belonging to an unseen template, is a challenging problem for many rea- sons. In contrast to most prior work on information extraction (Sarawagi, 2008), templatic documents do not contain much prose. Approaches that work well on natural text organized in sentences can- not be applied directly to such documents where spatial …

Content truncated to 500 words