Regularized Attentive Capsule Network for Overlapped Relation Extraction

Proceedings of the 28th International Conference on Computational Linguistics, pages 6388–6398 Barcelona, Spain (Online), December 8-13, 2020 6388 Regularized Attentive Capsule Network for Overlapped Relation Extraction Tianyi Liu1, Xiangyu Lin2, Weijia Jia3,1∗, Mingliang Zhou4, Wei Zhao5 1Department of Computer Science and Engineering, Shanghai Jiao Tong University 2Department of Computer and Information Science, University of Macau 3BNU-UIC Institute of AI and Future Networks, Beijing Normal University (Zhuhai) 4School of Computer Science, Chongqing University 5American University of Sharjah, Sharjah United Arab Emirates Abstract Distantly supervised relation extraction has been widely applied in knowledge base construction due to its less requirement of human efforts. However, the automatically established training datasets in distant supervision contain low-quality instances with noisy words and overlapped relations, introducing great challenges to the accurate extraction of relations. To address this problem, we propose a novel Regularized Attentive Capsule Network (RA-CapNet) to better identify highly overlapped relations in each informal sentence. To discover multiple relation features in an instance, we embed multi-head attention into the capsule network as the low-level capsules, where the subtraction of two entities acts as a new form of relation query to select salient features regardless of their positions. To further discriminate overlapped relation features, we devise disagreement regularization to explicitly encourage the diversity among both multiple attention heads and low-level capsules. Extensive experiments conducted on widely used datasets show that our model achieves significant improvements in relation extraction. 1 Introduction Relation extraction aims to extract relations between entities in text, where distant supervision proposed by (Mintz et al., 2009) automatically establishes training datasets by assigning relation labels to instances that mention entities within knowledge bases. However, the wrong labeling problem can occur and var- ious multi-instance learning methods (Riedel et al., 2010; Hoffmann et al., 2011; Surdeanu et al., 2012) have been proposed to address it. Despite the wrong labeling problem, each instance in distant super- vision is crawled from web pages, which is informal with many noisy words and can express multiple similar relations. This problem is not well-handled by previous approaches and severely hampers the performance of conventional neural relation extractors. To handle this problem, we have to address two challenges: (1) Identifying and gathering spotted relation information from low-quality instances; (2) Distinguishing multiple overlapped relation features from each instance. First, a few significant relation words are distributed dispersedly in the sentence, as shown in Figure 1, where words marked in red brackets represent entities, and italic words are key to expressing the rela- tions. For instance, the clause “evan bayh son of birch bayh” in S1 is sufficient to express the relation /people/person/children of evan bayh and birch bayh. Salient relation words are few in number and dis- persedly in S1, while others excluded from the clause can be regarded as noise. Traditional neural models have difficulty gathering spotted relation features at different positions along the sequence because they use Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN) as basic relation en- coders (Zeng et al., 2015; Liu et al., 2018; …

Content truncated to 500 words