Convolutional Neural Networks vs. Convolution Kernels: Feature Engineering for Answer Sentence Reranking

Proceedings of NAACL-HLT 2016, pages 1268–1278, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Convolutional Neural Networks vs. Convolution Kernels: Feature Engineering for Answer Sentence Reranking Kateryna Tymoshenko† and Daniele Bonadiman† and Alessandro Moschitti †DISI, University of Trento, 38123 Povo (TN), Italy Qatar Computing Research Institute, HBKU, 5825 Doha, Qatar {kateryna.tymoshenko,d.bonadiman}@unitn.it amoschitti@qf.org.qa Abstract In this paper, we study, compare and combine two state-of-the-art approaches to automatic feature engineering: Convolution Tree Ker- nels (CTKs) and Convolutional Neural Net- works (CNNs) for learning to rank answer sentences in a Question Answering (QA) set- ting. When dealing with QA, the key aspect is to encode relational information between the constituents of question and answer in learn- ing algorithms. For this purpose, we propose novel CNNs using relational information and combined them with relational CTKs. The results show that (i) both approaches achieve the state of the art on a question answering task, where CTKs produce higher accuracy and (ii) combining such methods leads to un- precedented high results. 1 Introduction The increasing use of machine learning for the de- sign of NLP applications pushes for fast methods for feature engineering. In contrast, the latter typically requires considerable effort especially when dealing with highly semantic tasks such as QA. For example, for an effective design of automated QA systems, the question text needs to be put in relation with the text passages retrieved from a document collection to enable an accurate extraction of the correct an- swers from passages. From a machine learning per- spective, encoding the information above consists in manually defining expressive rules and features based on syntactic and semantic patterns. Therefore, methods for automatizing feature en- gineering are remarkably important also in the light of fast prototyping of commercial applications. To the best of our knowledge, two of the most effec- tive methods for engineering features are: (i) kernel methods, which naturally map feature vectors or di- rectly objects in richer feature spaces; and more re- cently (ii) approaches based on deep learning, which have been shown to be very effective. Regarding the former, in (Moschitti et al., 2007), we firstly used CTKs in Support Vector Machines (SVMs) to generate features from a question (Q) and their candidate answer passages (AP). CTKs enable SVMs to learn in the space of convolutional subtrees of syntactic and semantic trees used for represent- ing Q and AP. This automatically engineers syntac- tic/semantic features. One important characteristic we added in (Severyn and Moschitti, 2012) is the use of relational links between Q and AP, which ba- sically merged the two syntactic trees in a relational graph (containing relational features). Although based on different principles, also CNNs can generate powerful features, e.g., see (Kalchbrenner et al., 2014; Kim, 2014). CNNs can effectively capture the compositional process of mapping the meaning of individual words in a sen- tence to a continuous representation of the sentence. This way CNNs can efficiently learn to embed input sentences into low-dimensional vector space, pre- serving important syntactic and …

Content truncated to 500 words