Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1753–1762, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks Daojian Zeng, Kang Liu, Yubo Chen and Jun Zhao National Laboratory of Pattern Recognition Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China {djzeng,kliu,yubo.chen,jzhao}@nlpr.ia.ac.cn Abstract Two problems arise when using distant su- pervision for relation extraction. First, in this method, an already existing knowl- edge base is heuristically aligned to texts, and the alignment results are treated as la- beled data. However, the heuristic align- ment can fail, resulting in wrong label problem. In addition, in previous ap- proaches, statistical models have typically been applied to ad hoc features. The noise that originates from the feature extraction process can cause poor performance. In this paper, we propose a novel model dubbed the Piecewise Convolu- tional Neural Networks (PCNNs) with multi-instance learning to address these two problems. To solve the first prob- lem, distant supervised relation extraction is treated as a multi-instance problem in which the uncertainty of instance labels is taken into account. To address the lat- ter problem, we avoid feature engineering and instead adopt convolutional architec- ture with piecewise max pooling to auto- matically learn relevant features. Exper- iments show that our method is effective and outperforms several competitive base- line methods. 1 Introduction In relation extraction, one challenge that is faced when building a machine learning system is the generation of training examples. One common technique for coping with this difficulty is distant supervision (Mintz et al., 2009) which assumes that if two entities have a relationship in a known knowledge base, then all sentences that mention these two entities will express that relationship in some way. Figure 1 shows an example of the auto- Freebase Mentions from free texts Figure 1: Training instances generated through distant supervision. Upper sentence: correct la- beling; lower sentence: incorrect labeling. matic labeling of data through distant supervision. In this example, Apple and Steve Jobs are two re- lated entities in Freebase1. All sentences that con- tain these two entities are selected as training in- stances. The distant supervision strategy is an ef- fective method of automatically labeling training data. However, it has two major shortcomings when used for relation extraction. First, the distant supervision assumption is too strong and causes the wrong label problem. A sen- tence that mentions two entities does not necessar- ily express their relation in a knowledge base. It is possible that these two entities may simply share the same topic. For instance, the upper sentence indeed expresses the “company/founders” relation in Figure 1. The lower sentence, however, does not express this relation but is still selected as a train- ing instance. This will hinder the performance of a model trained on such noisy data. Second, previous methods (Mintz et al., 2009; Riedel et al., 2010; Hoffmann et al., 2011) have typically applied supervised models to elaborately designed features when obtained …

Content truncated to 500 words