Document-level Relation Extraction with Dual-tier Heterogeneous Graph

Proceedings of the 28th International Conference on Computational Linguistics, pages 1630–1641 Barcelona, Spain (Online), December 8-13, 2020 1630 Document-level Relation Extraction with Dual-tier Heterogeneous Graph Zhenyu Zhang, Bowen Yu, Xiaobo Shu, Tingwen Liu∗, Hengzhu Tang, Yubin Wang and Li Guo Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China {zhangzhenyu1996,yubowen,shuxiaobo,liutingwen}@iie.ac.cn {tanghengzhu,wangyubin,guoli}@iie.ac.cn Abstract Document-level relation extraction (RE) poses new challenges over its sentence-level counterpart since it requires an adequate comprehension of the whole document and the multi-hop reasoning ability across multiple sentences to reach the final result. In this paper, we propose a novel graph- based model with Dual-tier Heterogeneous Graph (DHG) for document-level RE. In particular, DHG is composed of a structure modeling layer followed by a relation reasoning layer. The major advantage is that it is capable of not only capturing both the sequential and structural information of documents but also mixing them together to benefit for multi-hop reasoning and final decision- making. Furthermore, we employ Graph Neural Networks (GNNs) based message propagation strategy to accumulate information on DHG. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on two widely used datasets, and further analyses suggest that all the modules in our model are indispensable for document-level RE. 1 Introduction Relation extraction (RE), which aims to identify relational facts between entities from plain text, is one of the most fundamental tasks in information extraction (IE) and natural language processing (NLP). Existing methods usually focus on extracting relations from a single sentence (i.e., sentence-level RE) (Soares et al., 2019; Yu et al., 2019; Zhang et al., 2020a). However, sentence-level RE suffers from a non- ignorable limitation in practice: a large number of relational facts are expressed in multiple sentences. Lately, the attention of identifying inter-sentence relations heightens the interest of moving RE forward from sentence-level to document-level. The most straightforward way to perform document-level RE is to treat documents as long sequences and then employ sequential models, adapted from sentence-level RE, to extract the relation between given entities (Gu et al., 2017; Li et al., 2018a; Verga et al., 2018). However, these methods inevitably face several challenges in modeling long-term dependencies and enabling multi-hop reasoning, yet identify- ing long-term and inter-sentence relations is precisely the key and difficult point of document-level RE. Recently, a number of exquisite graph-based models for document-level NLP tasks show their sparkles in capturing multi-hop relations (De Cao et al., 2019; Tu et al., 2019; Christopoulou et al., 2019). Never- theless, these approaches often fail to adequately capture the inherent structure of documents and discard masses of valuable structural information when transforming documents into graphs. As a matter of fact, the document structure, especially the positional, syntactical, and hierarchical structure, has proven to be very effective for many document-level tasks (Miculicich et al., 2018; Li et al., 2018b). Look at a concrete example shown in Figure 1, in order to predict the relation between bradycardia and ramipril, one has to first …

Content truncated to 500 words