Can Taxonomy Help? Improving Semantic Question Matching using Question Taxonomy

Proceedings of the 27th International Conference on Computational Linguistics, pages 499–513 Santa Fe, New Mexico, USA, August 20-26, 2018. 499 Can Taxonomy Help? Improving Semantic Question Matching using Question Taxonomy Deepak Gupta∗, Rajkumar Pujari†, Asif Ekbal∗, Pushpak Bhattacharyya∗, Anutosh Maitra‡ Tom Jain‡ and Shubhashis Sengupta‡ ∗Indian Institute of Technology Patna, India †Purdue University, USA ‡Accenture Labs, Bengaluru, India ∗{deepak.pcs16,asif,pb}@iitp.ac.in, †rpujari@purdue.edu ‡{anutosh.maitra,tom.geo.jain,shubhashis.sengupta}@accenture.com Abstract In this paper, we propose a hybrid technique for semantic question matching. It uses a pro- posed two-layered taxonomy for English questions by augmenting state-of-the-art deep learning models with question classes obtained from a deep learning based question classifier. Experi- ments performed on three open-domain datasets demonstrate the effectiveness of our proposed approach. We achieve state-of-the-art results on partial ordering question ranking (POQR) bench- mark dataset. Our empirical analysis shows that coupling standard distributional features (pro- vided by the question encoder) with knowledge from taxonomy is more effective than either deep learning (DL) or taxonomy-based knowledge alone. 1 Introduction Question Answering (QA) is a well investigated research area in Natural Language Processing (NLP). There are several existing QA systems that answer factual questions with short answers (Iyyer et al., 2014; Bian et al., 2008; Ng and Kan, 2015). However, systems which attempt to answer questions that have long answers with several well-formed sentences, are rare in practice. This is mainly due to some of the following challenges: (i) selecting appropriate text fragments from document(s), (ii) gener- ating answer texts with coherent and cohesive sentences, (iii) ensuring the syntactic as well as semantic well-formedness of the answer text. However, when we already have a set of answered questions, re- constructing the answers for semantically similar questions can be bypassed. For each unseen question, the most semantically similar question is identified by comparing the unseen question with the existing set of questions. The question, which is closest to the unseen question can be retrieved as a possible semantically similar question. Thus, accurate semantic question matching can significantly improve a QA system. In the recent past, several deep learning based models such as recurrent neural networks (RNNs), convolution neural network (CNN), gated recurrent units (GRUs) etc. have been explored to obtain representation at the word (Mikolov et al., 2013; Pennington et al., 2014), sentence (Kim, 2014) and paragraph (Zhang et al., 2017) level. In the proposed semantic question matching framework, we use attention based neural network mod- els to generate question vectors. We create a hierarchical taxonomy by considering different types and subtypes in such a way that questions having similar answers belong to the same taxonomy class. We propose and train a deep learning based question classifier network to classify the taxonomy classes. The taxonomy information is helpful in taking a decision on semantic similarity between them. For exam- ple, the questions ‘How do scientists work?’ and ‘Where do scientists work?’, have very high lexical similarity but they have different answer types. This can be easily identified using a question taxonomy. Taxonomy can provide very useful information when we do …

Content truncated to 500 words