DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents

Proceedings of the 29th International Conference on Computational Linguistics, pages 2141–2146 October 12–17, 2022. 2141 DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents Mingfei Gao∗, Le Xue∗, Chetan Ramaiah, Chen Xing, Ran Xu, Caiming Xiong Salesforce Research, Palo Alto, USA {mingfei.gao, lxue, cramaiah, cxing, ran.xu, cxiong}@salesforce.com Abstract We propose, DocQueryNet, a value retrieval method with arbitrary queries for form-like doc- uments to reduce human effort of processing forms. Unlike previous methods that only ad- dress a fixed set of field items, our method pre- dicts target value for an arbitrary query based on the understanding of the layout and seman- tics of a form. To further boost model per- formance, we propose a simple document lan- guage modeling (SimpleDLM) strategy to im- prove document understanding on large-scale model pre-training. Experimental results show that DocQueryNet outperforms previous de- signs significantly and the SimpleDLM further improves our performance on value retrieval by around 17% F1 score compared with the state-of-the-art pre-training method. Code is available here. 1 Introduction Form-like documents are very commonly used in business workflows. However, tremendous forms are still processed manually everyday. When hu- mans need to extract some relevant information from a form-like document, they proceed as con- ducting a value retrieval task with some text queries. For example in Figure 1, when humans process a form, they usually have a description of the infor- mation (query) that they want to extract (e.g., total page number). Then, they examine the form (usu- ally an image or a PDF) carefully to locate the key (e.g., NUMBER OF PAGES INCLUDING COVER SHEET in Figure 1) that is most semantically sim- ilar to the query and finally infer the target value based on the localized key. This manual process costs a large amount of human efforts as the num- ber of forms and queries increases. Automating information extraction from forms is important to alleviate this problem. ∗Mingfei and Le contributed equally. key value Query: total page number Target value: 3 Figure 1: Illustration of the value retrieval with an ar- bitrary query. A user obtains the target value from a document based on a query of interest. Some values are decorated with a mosaic for privacy purposes. Existing methods formulate the problem as se- quence labeling (Xu et al., 2020b) or field extrac- tion (Gao et al., 2022), where they define a fixed set of items of interest (referred as fields) and train models that only extract values of the pre-defined fields. There are at least two limitations of this formulation. First, forms are very diverse and it is impossible to cover all the items of interest using a fixed set of fields. Second, their models are very domain-specific and hard to be utilized for different form types. For example, an invoice field extractor may not be able to process resumes, since different fields are expected for these two form types. To handle diverse queries with a unified model, we formulate the problem as value retrieval with arbitrary queries for …

Content truncated to 500 words