Graph Enhanced Dual Attention Network for Document-Level Relation Extraction

Proceedings of the 28th International Conference on Computational Linguistics, pages 1551–1560 Barcelona, Spain (Online), December 8-13, 2020 1551 Graph Enhanced Dual Attention Network for Document-Level Relation Extraction Bo Li1,2, Wei Ye1∗, Zhonghao Sheng1,2, Rui Xie1,2, Xiangyu Xi1,2, Shikun Zhang1 1National Engineering Research Center for Software Engineering, Peking University 2School of Software and Microelectronics, Peking University deepblue.lb@stu.pku.edu.cn, {wye, zhonghao.sheng, ruixie, xixy, zhangsk}@pku.edu.cn Abstract Document-level relation extraction requires inter-sentence reasoning capabilities to capture lo- cal and global contextual information for multiple relational facts. To improve inter-sentence reasoning, we propose to characterize the complex interaction between sentences and potential relation instances via a Graph Enhanced Dual Attention network (GEDA). In GEDA, sentence representation generated by the sentence-to-relation (S2R) attention is refined and synthesized by a Heterogeneous Graph Convolutional Network before being fed into the relation-to-sentence (R2S) attention . We further design a simple yet effective regularizer based on the natural duality of the S2R and R2S attention, whose weights are also supervised by the supporting evidence of relation instances during training. An extensive set of experiments on an existing large-scale dataset show that our model achieves competitive performance, especially for the inter-sentence relation extraction, while the neural predictions can also be interpretable and easily observed. 1 Introduction Relation extraction (RE) is an important research topic in natural language processing with plenty of applications. The task of RE is to detect the relationship from a given context and target entities. De- pending on the given context, RE can be divided into two types: sentence-level RE (Zeng et al., 2015; Zhou et al., 2016; Ji et al., 2017; He et al., 2018; Fu et al., 2019; Guo et al., 2019; Ye et al., 2019) and document-level RE (Sahu et al., 2019; Gupta et al., 2019; Christopoulou et al., 2019; Wang et al., 2019). Compared with widely-studied sentence-level RE, document-level RE is a more challenging task that requires more investigations. One of main challenges in document-level RE is how to perform inter-sentence reasoning over a long document to synthesize local and global information for potential relation facts. According to the statistics of DocRed (Yao et al., 2019), a large-scale human-annotated document-level RE dataset, there are 40.7% relational facts can only be extracted from multiple sentences in this dataset. That means a desirable neural model for document-level RE requires sophisticated inter-sentence reasoning capabilities. In this paper, we propose to improve inter-sentence reasoning from a perspective of better character- izing the complex interaction between multiple sentences and multiple potential relation instances in a document. Since one relation instance may be expressed by multiple sentences and one sentence may re- veal multiple relational facts (or parts of relational facts), it is natural and straightforward to use attention between sentences and potential relation instances to capture the complex many-to-many interaction. To this end, we introduce a bi-directional attention mechanism consisting of the attention paid by a sentence to relation instances (sentence-to-relation, S2R) and the attention paid by a relation instance to sentences (relation-to-sentence, R2S). Though bi-directional attention has been widely …

Content truncated to 500 words