Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Paths

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1785–1794, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Paths Yan Xu,† Lili Mou,† Ge Li,†∗Yunchuan Chen,‡ Hao Peng,† Zhi Jin†∗ †Software Institute, Peking University, 100871, P. R. China {xuyan14,lige,zhijin}@sei.pku.edu.cn,{doublepower.mou,penghao.pku}@gmail.com ‡University of Chinese Academy of Sciences, chenyunchuan11@mails.ucas.ac.cn Abstract Relation classification is an important re- search arena in the field of natural lan- guage processing (NLP). In this paper, we present SDP-LSTM, a novel neural net- work to classify the relation of two enti- ties in a sentence. Our neural architecture leverages the shortest dependency path (SDP) between two entities; multichan- nel recurrent neural networks, with long short term memory (LSTM) units, pick up heterogeneous information along the SDP. Our proposed model has several dis- tinct features: (1) The shortest dependency paths retain most relevant information (to relation classification), while eliminating irrelevant words in the sentence. (2) The multichannel LSTM networks allow ef- fective information integration from het- erogeneous sources over the dependency paths. (3) A customized dropout strategy regularizes the neural network to allevi- ate overfitting. We test our model on the SemEval 2010 relation classification task, and achieve an F1-score of 83.7%, higher than competing methods in the literature. 1 Introduction Relation classification is an important NLP task. It plays a key role in various scenarios, e.g., in- formation extraction (Wu and Weld, 2010), ques- tion answering (Yao and Van Durme, 2014), med- ical informatics (Wang and Fan, 2014), ontol- ogy learning (Xu et al., 2014), etc. The aim of relation classification is to categorize into pre- defined classes the relations between pairs of marked entities in given texts. For instance, in the sentence “A trillion gallons of [water]e1 have been poured into an empty [region]e2 of outer ∗Corresponding authors. space,” the entities water and region are of rela- tion Entity-Destination(e1, e2). Traditional relation classification approaches rely largely on feature representation (Kambhatla, 2004), or kernel design (Zelenko et al., 2003; Bunescu and Mooney, 2005). The former method usually incorporates a large set of features; it is difficult to improve the model performance if the feature set is not very well chosen. The latter ap- proach, on the other hand, depends largely on the designed kernel, which summarizes all data infor- mation. Deep neural networks, emerging recently, provide a way of highly automatic feature learning (Bengio et al., 2013), and have exhibited consid- erable potential (Zeng et al., 2014; dos Santos et al., 2015). However, human engineering—that is, incorporating human knowledge to the network’s architecture—is still important and beneficial. This paper proposes a new neural network, SDP-LSTM, for relation classification. Our model utilizes the shortest dependency path (SDP) be- tween two entities in a sentence; we also design a long short term memory (LSTM)-based recurrent neural network for information processing. The neural architecture is mainly inspired by the fol- lowing observations. • Shortest dependency paths are informative (Fundel et al., 2007; Chen et al., …

Content truncated to 500 words