Document-level Entity-based Extraction as Template Generation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5257–5269 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5257 Document-level Entity-based Extraction as Template Generation Kung-Hsiang Huang1,2 Sam Tang3 Nanyun Peng1,3 1 Information Sciences Institute, University of Southern California 2 Department of Computer Science, University of Illinois Urbana-Champaign 3 Computer Science Department, University of California, Los Angeles khhuang3@illinois.edu samtang1430@gmail.com violetpeng@cs.ucla.edu Abstract Document-level entity-based extraction (EE), aiming at extracting entity-centric informa- tion such as entity roles and entity relations, is key to automatic knowledge acquisition from text corpora for various domains. Most document-level EE systems build extractive models, which struggle to model long-term dependencies among entities at the document level. To address this issue, we propose a generative framework for two document-level EE tasks: role-filler entity extraction (REE) and relation extraction (RE). We first formu- late them as a template generation problem, allowing models to efficiently capture cross- entity dependencies, exploit label semantics, and avoid the exponential computation com- plexity of identifying N-ary relations. A novel cross-attention guided copy mechanism, TOPK COPY, is incorporated into a pre-trained sequence-to-sequence model to enhance the capabilities of identifying key information in the input document. Experiments done on the MUC-4 and SCIREX dataset show new state- of-the-art results on REE (+3.26%), binary RE (+4.8%), and 4-ary RE (+2.7%) in F1 score 1. 1 Introduction Document-level entity-based extraction (EE) are tasks that extract entity-centric information, such as entities and their relations, from unstructured text across multiple sentences. With the rise of big data in recent years, document-level EE is growing in importance with applications such as understand- ing clinical reports (Nye et al., 2020), extracting document-level events (Huang and Peng, 2021), and building knowledge graphs from journals (Wu et al., 2020). In this work, we focus on two clas- sic tasks of document-level EE: role-filler entity extraction (REE) and relation extraction (RE). Recent works on document-level EE usually build 1The source code is publicly available at https:// github.com/PlusLabNLP/TempGen Ours SciREX-P Gold Relations SNLI-accuracy We conducted experiments on the Stanford Natural Language Inference ( SNLI ) dataset ... (39 sentences) ... Our best score on this task is 87.3 % accuracy. ... (20 sentences ) ... we adopt MAP and MRR as the evaluation metrics for this task. SNLI-accuracy SNLI-MAP SNLI-MRR Figure 1: A comparison between our approach and a competitive extractive system, SCIREX-P (Jain et al., 2020), on a relation extraction example from SCIREX. The task is to extract entities and identify which entities are related from the given scientific article. Due to the long distances between entities, SCIREX-P struggles to extract the right entity pair that has a relation, while our approach correctly identifies them. This reflects our method’s advantage in modeling long-term cross-entity dependencies. task-specific classifiers on top of large pre-trained language models. For example, Du and Cardie (2020a) builds a sequence tagging framework with multi-granularity representations based on BERT (Devlin et al., 2019) for role-filler entity extrac- tion. Jain et al. (2020) builds a relation extrac- tion pipeline upon …

Content truncated to 500 words