Neural Relation Extraction with Selective Attention over Instances

Neural Relation Extraction with Selective Attention over Instances Yankai Lin1, Shiqi Shen1, Zhiyuan Liu1,2∗, Huanbo Luan1, Maosong Sun1,2 1 Department of Computer Science and Technology, State Key Lab on Intelligent Technology and Systems, National Lab for Information Science and Technology, Tsinghua University, Beijing, China 2 Jiangsu Collaborative Innovation Center for Language Competence, Jiangsu, China Abstract Distant supervised relation extraction has been widely used to find novel relational facts from text. However, distant su- pervision inevitably accompanies with the wrong labelling problem, and these noisy data will substantially hurt the perfor- mance of relation extraction. To allevi- ate this issue, we propose a sentence-level attention-based model for relation extrac- tion. In this model, we employ convolu- tional neural networks to embed the se- mantics of sentences. Afterwards, we build sentence-level attention over multi- ple instances, which is expected to dy- namically reduce the weights of those noisy instances. Experimental results on real-world datasets show that, our model can make full use of all informative sen- tences and effectively reduce the influence of wrong labelled instances. Our model achieves significant and consistent im- provements on relation extraction as com- pared with baselines. The source code of this paper can be obtained from https: //github.com/thunlp/NRE. 1 Introduction In recent years, various large-scale knowledge bases (KBs) such as Freebase (Bollacker et al., 2008), DBpedia (Auer et al., 2007) and YAGO (Suchanek et al., 2007) have been built and widely used in many natural language processing (NLP) tasks, including web search and question answer- ing. These KBs mostly compose of relational facts with triple format, e.g., (Microsoft, founder, Bill Gates). Although existing KBs contain a ∗ Corresponding author: Zhiyuan Liu (li- uzy@tsinghua.edu.cn). massive amount of facts, they are still far from complete compared to the infinite real-world facts. To enrich KBs, many efforts have been invested in automatically finding unknown relational facts. Therefore, relation extraction (RE), the process of generating relational data from plain text, is a cru- cial task in NLP. Most existing supervised RE systems require a large amount of labelled relation-specific training data, which is very time consuming and labor in- tensive. (Mintz et al., 2009) proposes distant su- pervision to automatically generate training data via aligning KBs and texts. They assume that if two entities have a relation in KBs, then all sen- tences that contain these two entities will express this relation. For example, (Microsoft, founder, Bill Gates) is a relational fact in KB. Distant su- pervision will regard all sentences that contain these two entities as active instances for relation founder. Although distant supervision is an effective strategy to automatically label training data, it always suffers from wrong labelling prob- lem. For example, the sentence “Bill Gates ’s turn to philanthropy was linked to the antitrust prob- lems Microsoft had in the U.S. and the European union.” does not express the relation founder but will still be regarded as an active instance. Hence, (Riedel et al., 2010; Hoffmann et al., 2011; Surdeanu et al., 2012) adopt multi-instance learn- ing …

Content truncated to 500 words