Learning Logic Rules for Document-Level Relation Extraction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1239–1250 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1239 Learning Logic Rules for Document-level Relation Extraction Dongyu Ru†‡, Changzhi Sun‡∗, Jiangtao Feng‡, Lin Qiu†, Hao Zhou‡, Weinan Zhang†∗, Yong Yu†, Lei Li§† †Shanghai Jiao Tong University ‡ByteDance AI Lab §University of California, Santa Barbara {maxru,lqiu,wnzhang,yyu}@apex.sjtu.edu.cn {sunchangzhi,fengjiangtao,zhouhao.nlp}@bytedance.com lilei@cs.ucsb.edu Abstract Document-level relation extraction aims to identify relations between entities in a whole document. Prior efforts to capture long-range dependencies have relied heavily on implic- itly powerful representations learned through (graph) neural networks, which makes the model less transparent. To tackle this chal- lenge, in this paper, we propose LogiRE, a novel probabilistic model for document-level relation extraction by learning logic rules. Lo- giRE treats logic rules as latent variables and consists of two modules: a rule generator and a relation extractor. The rule generator is to generate logic rules potentially contributing to final predictions, and the relation extrac- tor outputs final predictions based on the gen- erated logic rules. Those two modules can be efficiently optimized with the expectation- maximization (EM) algorithm. By introduc- ing logic rules into neural networks, LogiRE can explicitly capture long-range dependen- cies as well as enjoy better interpretation. Empirical results show that LogiRE signifi- cantly outperforms several strong baselines in terms of relation performance (∼1.8 F1 score) and logical consistency (over 3.3 logic score). Our code is available at https://github. com/rudongyu/LogiRE. 1 Introduction Extracting relations from a document has attracted significant research attention in information extrac- tion (IE). Recently, instead of focusing on sentence- level (Socher et al., 2012; dos Santos et al., 2015; Han et al., 2018; Zhang et al., 2018; Wang et al., 2021a,b), researchers have turned to modeling di- rectly at the document level (Wang et al., 2019; Ye et al., 2020; Zhou et al., 2021), which provides longer context and requires more complex reason- ing. Early efforts focus mainly on learning a power- ful relation (i.e., entity pair) representation, which ∗corresponding authors. †Work is done while at ByteDance. Figure 1: An example of relation identification by utilizing rules. The three labeled sentences describe the relations royalty_of(Harry,UK), sib- ling_of(William,Harry), and spouse_of(Kate,William), respectively. The identification of the relation roy- alty_of(Kate,UK) requires the synthesis of information in three sentences. It can be easily derived from the demonstrated rule and the other three relations. implicitly captures long-range dependencies. Ac- cording to the input structure, we can divide the existing document-level relation extraction work into two categories: the sequence-based model and the graph-based model. The sequence-based model first leverages differ- ent sequence encoder (e.g., BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019)) to obtain token representations, and then computes relation repre- sentations by various pooling operations, e.g., aver- age pooling (Yao et al., 2019; Xu et al., 2021), at- tentive pooling (Zhou et al., 2021). To further cap- ture long-range dependencies, graph-based mod- els are proposed. By constructing a graph, words or entities that are far away can become neighbor nodes. On …

Content truncated to 500 words