Attention-Based Convolutional Neural Network for Semantic Relation Extraction

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2526–2536, Osaka, Japan, December 11-17 2016. Attention-Based Convolutional Neural Network for Semantic Relation Extraction Yatian Shen, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, P.R.China {10110240031,xjhuang}@fudan.edu.cn Abstract Nowadays, neural networks play an important role in the task of relation classification. In this paper, we propose a novel attention-based convolutional neural network architecture for this task. Our model makes full use of word embedding, part-of-speech tag embedding and position embedding information. Word level attention mechanism is able to better determine which parts of the sentence are most influential with respect to the two entities of interest. This architecture enables learning some important features from task-specific labeled data, forgoing the need for external knowledge such as explicit dependency structures. Experiments on the SemEval-2010 Task 8 benchmark dataset show that our model achieves better performances than several state- of-the-art neural network models and can achieve a competitive performance just with minimal feature engineering. 1 Introduction Classifying the relation between two entities in a given context is an important task in natural language processing (NLP). Take the following sentence as an example: Jewelry and other smaller ⟨e1⟩ valuables ⟨/e1⟩ were locked in a ⟨e2⟩ safe ⟨/e2⟩ or a closet with a dead-bolt. Here, the marked entities “valuables” and “safe” are of the relation “Content-Container(e1; e2)”. Relation classification plays a key role in various NLP applications, and has become a hot research topic in recent years. Various machine learning based relation classification methods have been proposed for the task, based on either human-designed features (Kambhatla, 2004; Suchanek et al., 2006), or kernels (Kambhatla, 2004; Suchanek et al., 2006). Some researchers also employed the existing known facts to label the text corpora via distant supervision (Mintz et al., 2009; Riedel et al., 2010; Hoffmann et al., 2011; Takamatsu et al., 2012). All of these approaches are effective because they leverage a large body of linguistic knowledge. However, these methods may suffer from two limitations. First, the extracted features or elaborately designed kernels are often derived from the output of pre-existing NLP systems, which leads to the propagation of the errors in the existing tools and hinders the performance of such systems (Bach and Badaskar, 2007). Second, the methods mentioned above do not scale well during relation extraction, which makes it very hard to engineer effective task-specific features and learn parameters. Recently, neural network models have been increasingly focused on for their ability to minimize the effort in feature engineering of NLP tasks (Collobert et al., 2011; Zheng et al., 2013; Pei et al., 2014). Moreover, some researchers have also paid attention to feature learning of neural networks in the field of relation extraction. (Socher et al., 2012) introduced a recursive neural network model to learn compo- sitional vector representations for phrases and sentences of arbitrary syntactic types and length. (Zeng et al., 2014; Xu et al., 2015b) utilized convolutional neural networks …

Content truncated to 500 words