Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions

Proceedings of NAACL-HLT 2019, pages 2810–2819 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2810 Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions Zhi-Xiu Ye University of Science and Technology of China zxye@mail.ustc.edu.cn Zhen-Hua Ling University of Science and Technology of China zhling@ustc.edu.cn Abstract This paper presents a neural relation extrac- tion method to deal with the noisy training data generated by distant supervision. Pre- vious studies mainly focus on sentence-level de-noising by designing neural networks with intra-bag attentions. In this paper, both intra- bag and inter-bag attentions are considered in order to deal with the noise at sentence-level and bag-level respectively. First, relation- aware bag representations are calculated by weighting sentence embeddings using intra- bag attentions. Here, each possible relation is utilized as the query for attention calcula- tion instead of only using the target relation in conventional methods. Furthermore, the rep- resentation of a group of bags in the training set which share the same relation label is cal- culated by weighting bag representations us- ing a similarity-based inter-bag attention mod- ule. Finally, a bag group is utilized as a train- ing sample when building our relation extrac- tor. Experimental results on the New York Times dataset demonstrate the effectiveness of our proposed intra-bag and inter-bag attention modules. Our method also achieves better re- lation extraction accuracy than state-of-the-art methods on this dataset1. 1 Introduction Relation Extraction is a fundamental task in nat- ural language processing (NLP), which aims to extract semantic relations between entities. For example, sentence “[Barack Obama]e1 was born in [Hawaii]e2” expresses the relation BornIn be- tween entity pair Barack Obama and Hawaii. Conventional relation extraction methods, such as (Zelenko et al., 2002; Culotta and Sorensen, 2004; Mooney and Bunescu, 2006), adopted su- pervised training and suffered from the lack of 1The code is available at https://github.com/ZhixiuYe/ Intra-Bag-and-Inter-Bag-Attentions. bag sentence correct? B1 S1. Barack Obamna was born in the United States. Yes S2. Barack Obamna was the 44th president of the United States No B2 S3. Kyle Busch , a Las Vegas res- ident who ran second to Johnson last year, finished third, followed by Kasey Kahne, Jeff Gordon and mark martin . No S4. Hendrick drivers finished in three of the top four spots at Las Vegas , in- cluding Kyle Busch in second and ... No Table 1: Examples of sentences with relation place of birth annotated by distant supervision, where “Yes” and “No” indicate whether or not each sentence actually expresses this relation. large-scale manually labeled data. To address this issue, the distant supervision method (Mintz et al., 2009) was proposed, which generated the data for training relation extraction models automatically. The distant supervision assumption says that if two entities participate in a relation, all sentences that mention these two entities express that rela- tion. It is inevitable that there exists noise in the data labeled by distant supervision. For example, the precision of aligning the relations in Freebase to the New York Times corpus was …

Content truncated to 500 words