A Position Encoding Convolutional Neural Network Based on Dependency Tree for Relation Classification

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 65–74, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics A Position Encoding Convolutional Neural Network Based on Dependency Tree for Relation Classification Yunlun Yang Yunhai Tong∗ Shulei Ma Zhi-Hong Deng∗ {incomparable-lun, yhtong, mashulei, zhdeng}@pku.edu.cn Key Laboratory of Machine Perception (Ministry of Education), School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China Abstract With the renaissance of neural network in re- cent years, relation classification has again become a research hotspot in natural lan- guage processing, and leveraging parse trees is a common and effective method of tack- ling this problem. In this work, we offer a new perspective on utilizing syntactic infor- mation of dependency parse tree and present a position encoding convolutional neural net- work (PECNN) based on dependency parse tree for relation classification. First, tree- based position features are proposed to en- code the relative positions of words in depen- dency trees and help enhance the word repre- sentations. Then, based on a redefinition of “context”, we design two kinds of tree-based convolution kernels for capturing the semantic and structural information provided by depen- dency trees. Finally, the features extracted by convolution module are fed to a classifier for labelling the semantic relations. Experiments on the benchmark dataset show that PECNN outperforms state-of-the-art approaches. We also compare the effect of different position features and visualize the influence of tree- based position feature by tracing back the con- volution process. 1 Introduction Relation classification focuses on classifying the se- mantic relations between pairs of marked entities in given sentences (Hendrickx et al., 2010). It is a fun- damental task which can serve as a pre-existing sys- tem and provide prior knowledge for information ex- ∗Corresponding authors traction, natural language understanding, informa- tion retrieval, etc. However, automatic recognition of semantic relation is challenging. Traditional fea- ture based approaches rely heavily on the quantity and quality of hand-crafted features and lexical re- sources, and it is time-consuming to select an op- timal subset of relevant features in order to maxi- mize performance. Though kernel based methods get rid of the feature selection process, they need elaborately designed kernels and are also computa- tionally expensive. Recently, with the renaissance of neural network, deep learning techniques have been adopted to pro- vide end-to-end solutions for many classic NLP tasks, such as sentence modeling (Socher, 2014; Kim, 2014) and machine translation (Cho et al., 2014). Recursive Neural Network (RNN) (Socher et al., 2012) and Convolutional Neural Network (CNN) (Zeng et al., 2014) have proven powerful in relation classification. In contrast to traditional approaches, neural network based methods own the ability of automatic feature learning and alleviate the problem of severe dependence on human-designed features and kernels. However, previous researches (Socher et al., 2012) imply that some features exploited by tradi- tional methods are still informative and can help en- hance the performance of neural network in relation classification. One simple but effective approach is to concatenate …

Content truncated to 500 words