Learning to Prune Dependency Trees with Rethinking for Neural Relation Extraction

Proceedings of the 28th International Conference on Computational Linguistics, pages 3842–3852 Barcelona, Spain (Online), December 8-13, 2020 3842 Learning to Prune Dependency Trees with Rethinking for Neural Relation Extraction Bowen Yu1,2, Mengge Xue1,2, Zhenyu Zhang1,2, Tingwen Liu1,2∗, Yubin Wang1,2 and Bin Wang3 1Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China 2School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China 3Xiaomi AI Lab, Xiaomi Inc., Beijing, China {yubowen,xuemengge,zhangzhenyu1996}@iie.ac.cn {liutingwen,wangyubin}@iie.ac.cn wangbin11@xiaomi.com Abstract Dependency trees have been shown to be effective in capturing long-range relations between tar- get entities. Nevertheless, how to selectively emphasize target-relevant information and remove irrelevant content from the tree is still an open problem. Existing approaches employing pre- defined rules to eliminate noise may not always yield optimal results due to the complexity and variability of natural language. In this paper, we present a novel architecture named Dynami- cally Pruned Graph Convolutional Network (DP-GCN), which learns to prune the dependency tree with rethinking in an end-to-end scheme. In each layer of DP-GCN, we employ a selection module to concentrate on nodes expressing the target relation by a set of binary gates and then augment the pruned tree with a pruned semantic graph to ensure the connectivity. After that, we introduce a rethinking mechanism to guide and refine the pruning operation by feeding back the high-level learned features repeatedly. Extensive experimental results demonstrate that our model achieves impressive performance compared to strong competitors. 1 Introduction Relation extraction (RE) aims to detect the semantic relationship between two specific entities appear- ing in a sentence (often termed subject and object, respectively). This task plays an important role in many downstream NLP applications that require a relational understanding of unstructured text such as question answering (Dai et al., 2016) and dialogue systems (Young et al., 2018). Models leveraging the dependency tree of the input sentence have proven to be effective in relation extraction because they can effortlessly exploit long-term relations that are obscure from the surface form (Zhang et al., 2018; Can et al., 2019). Recent studies also stated that not all tokens in the depen- dency tree are needed to express the relation of the target entity pair (Xu et al., 2015b; Zhang et al., 2018), and some target-irrelevant tokens could introduce noise and cause confusion to the classification. Therefore, multiple pruning strategies are proposed to eliminate unimportant tokens and distill the de- pendency information. Xu et al. (2015b) applied neural networks only on the shortest dependency path (SDP) between the two entities in the dependency tree, which soon became dominant with many works demonstrating that using SDP brings better experimental results than using the whole sentence (Xu et al., 2015a; Cai et al., 2016). Miwa and Bansal (2016) reduced the full tree to the subtree below the lowest common ancestor (LCA) of the entities. Zhang et al. (2018) expanded SDP by including tokens that are up to distance K away from the dependency path in the LCA subtree. However, these hand-crafted prun- ing rules may lead …

Content truncated to 500 words