Document-Level Relation Extraction with Sentences Importance Estimation and Focusing

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2920 - 2929 July 10-15, 2022 ©2022 Association for Computational Linguistics Document-Level Relation Extraction with Sentences Importance Estimation and Focusing Wang Xu1, Kehai Chen1, Lili Mou2, Tiejun Zhao1 1Harbin Institute of Technology, China 2Dept. Computing Science, Alberta Machine Intelligence Institute (Amii) University of Alberta, Canada xuwang@hit-mtlab.net, {chenkehai,tjzhao}@hit.edu.cn, doublepower.mou@gmail.com Abstract Document-level relation extraction (DocRE) aims to determine the relation between two entities from a document of multiple sentences. Recent studies typically represent the entire document by sequence- or graph-based models to predict the relations of all entity pairs. However, we find that such a model is not robust and exhibits bizarre behaviors: predicts correctly when an entire test documentit is fed as input, but errs when non-evidence sentences are removed. To this end, we propose a Sentence Importance Estimation and Focusing (SIEF) framework for DocRE, where we design a sentence importance score and a sentence focusing loss, encouraging DocRE models to focus on evidence sentences. Experimental results on two domains show that our SIEF not only improves overall performance, but also makes DocRE models more robust. Moreover, SIEF is a general framework, shown to be effective when combined with a variety of base DocRE models.1 1 Introduction Document-level relation extraction (DocRE) aims to predict entity relations across multiple sentences. It plays a crucial role in a variety of knowledge- based applications, such as question answer- ing (Sorokin and Gurevych, 2017) and large-scale knowledge graph construction (Baldini Soares et al., 2019). Different from sentence-level relation extraction (Zeng et al., 2014; Xiao and Liu, 2016; Song et al., 2019), the supporting evidence in the DocRE setting may involve multiple sentences scattering in the document. Thus, DocRE is more a realistic setting, attracting increasing attention in the field of information extraction. Most recent DocRE studies use the entire document as a clue to predict the relations of 1The code is publicly available at https://github. com/xwjim/SIEF Model Input {1,2,3} {1,2} Ground Truth GAIN Prediction MemberOf MemberOf MemberOf not MemberOf (undesired) [1] Rage Against the Machine is an American rap metal band from Los Angeles, California. [2] Formed in 1991, the group consists of vocalist Zack de la Rocha, guitarist Tom Morello, bassist Tim Commerford and drummer Brad Wilk. [3] After a self-issued demo, the band signed with Epic Records and released its debut album Rage Against the Machine in 1992. … Relation: MemberOf Supporting Evidence: {1,2} Figure 1: A DocRE model predicts correctly for an entire document, but errs when a non-evidence sentence is removed. all entity pairs without concerning where the evidence is located (Nan et al., 2020; Zeng et al., 2020; Xu et al., 2021a,b). However, one can identify the relation of a specific entity pair from a few sentences. Huang et al. (2021) show that irrelevant sentences in the document would hinder the performance of the model. Moreover, we observe that a DocRE model, trained on the entire document, may err when …

Content truncated to 500 words