Semantic Relation Classification via Hierarchical Recurrent Neural Network with Attention

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1254–1263, Osaka, Japan, December 11-17 2016. Semantic Relation Classification via Hierarchical Recurrent Neural Network with Attention Minguang Xiao Cong Liu∗ School of Data and Computer Science, Sun Yat-sen University xiaomg@mail2.sysu.edu.cn, liucong3@mail.sysu.edu.cn Abstract Semantic relation classification remains a challenge in natural language processing. In this pa- per, we introduce a hierarchical recurrent neural network that is capable of extracting informa- tion from raw sentences for relation classification. Our model has several distinctive features: (1) Each sentence is divided into three context subsequences according to two annotated nom- inals, which allows the model to encode each context subsequence independently so as to se- lectively focus as on the important context information; (2) The hierarchical model consists of two recurrent neural networks (RNNs): the first one learns context representations of the three context subsequences respectively, and the second one computes semantic composition of these three representations and produces a sentence representation for the relationship classification of the two nominals. (3) The attention mechanism is adopted in both RNNs to encourage the model to concentrate on the important information when learning the sentence representations. Experi- mental results on the SemEval-2010 Task 8 dataset demonstrate that our model is comparable to the state-of-the-art without using any hand-crafted features. 1 Introduction Semantic relation classification is an important task in natural language processing, which has attracted great attention in recent years. The goal is to identify the semantic relationship between a pair of nominals marked in a sentence. For instance, in the sentence “The software [company]e1 addressed the problem with the [publication]e2 of a fix on Saturday”, the marked nominals of company and publication are of relationship Product-Producer(e2, e1). Most conventional models focus on machine learning and feature design, which have been shown to obtain performance improvements (Kambhatla, 2004; Tratz and Hovy, 2010; Rink and Harabagiu, 2010). Recently, neural network approaches have been widely used for relation classification, which aim at reducing the need of hand-crafted features. These approaches are broadly divided into two categories: one explores the effectiveness of using dependency paths and its attached subtrees between two nominals, and various neural networks are adopted to model the shortest dependency paths and dependency sub- trees (Xu et al., 2015a; Xu et al., 2015b; Liu et al., 2015); the other exploits deep neural networks to learn syntactic and semantic features from raw sentences (Zeng et al., 2014; Dos Santos et al., 2015; Zhang et al., 2015), which has been proved effective, but inevitably suffers from irrelevant parts. Our paper introduces an attentive neural network that selectively focuses on useful information on raw sentences. Context information of the annotated nominals has been widely believed to be useful for relation clas- sification (Zhang et al., 2015; Thang Vu et al., 2016). In this work, we further explore the effectiveness of context information around the annotated nominals in a sentence. In our model, a sentence with two marked nominals is divided into three context subsequences according …

Content truncated to 500 words