Combining Recurrent and Convolutional Neural Networks for Relation Classification

Proceedings of NAACL-HLT 2016, pages 534–539, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Combining Recurrent and Convolutional Neural Networks for Relation Classification Ngoc Thang Vu1,2 and Heike Adel1 and Pankaj Gupta3 and Hinrich Sch¨utze1 1Center for Information and Language Processing, LMU Munich Oettingenstr. 67, 80538 Munich, Germany 2Institute for Natural Language Processing, University of Stuttgart Pfaffenwaldring 5b, 70569 Stuttgart, Germany 3Siemens Corporate Technology - Knowledge Modeling and Retrieval Otto-Hahn-Ring 6, 81739 Munich, Germany thang.vu@ims.uni-stuttgart.de | heike.adel@cis.lmu.de gupta.pankaj.ext@siemens.com | inquiries@cislmu.org Abstract This paper investigates two different neural architectures for the task of relation classi- fication: convolutional neural networks and recurrent neural networks. For both mod- els, we demonstrate the effect of different ar- chitectural choices. We present a new con- text representation for convolutional neural networks for relation classification (extended middle context). Furthermore, we propose connectionist bi-directional recurrent neural networks and introduce ranking loss for their optimization. Finally, we show that combin- ing convolutional and recurrent neural net- works using a simple voting scheme is accu- rate enough to improve results. Our neural models achieve state-of-the-art results on the SemEval 2010 relation classification task. 1 Introduction Relation classification is the task of assigning sentences with two marked entities to a prede- fined set of relations. The sentence “We poured the <e1>milk</e1> into the <e2>pumpkin mix- ture</e2>.”, for example, expresses the relation Entity-Destination(e1,e2). While early research mostly focused on support vector ma- chines or maximum entropy classifiers (Rink and Harabagiu, 2010a; Tratz and Hovy, 2010), recent research showed performance improvements by ap- plying neural networks (NNs) (Socher et al., 2012; Zeng et al., 2014; Yu et al., 2014; Nguyen and Gr- ishman, 2015; Dos Santos et al., 2015; Zhang and Wang, 2015) on the benchmark data from SemEval 2010 shared task 8 (Hendrickx et al., 2010) . This study investigates two different types of NNs: recurrent neural networks (RNNs) and con- volutional neural networks (CNNs) as well as their combination. We make the following contributions: (1) We propose extended middle context, a new context representation for CNNs for relation classi- fication. The extended middle context uses all parts of the sentence (the relation arguments, left of the relation arguments, between the arguments, right of the arguments) and pays special attention to the mid- dle part. (2) We present connectionist bi-directional RNN models which are especially suited for sentence clas- sification tasks since they combine all intermediate hidden layers for their final decision. Furthermore, the ranking loss function is introduced for the RNN model optimization which has not been investigated in the literature for relation classification before. (3) Finally, we combine CNNs and RNNs using a simple voting scheme and achieve new state-of-the- art results on the SemEval 2010 benchmark dataset. 2 Related Work In 2010, manually annotated data for relation clas- sification was released in the context of a SemEval shared task (Hendrickx et al., 2010). Shared task participants used, i.a., support vector machines or maximum entropy classifiers (Rink and Harabagiu, 2010a; Tratz and Hovy, 2010). Recently, their …

Content truncated to 500 words