Integrating Question Classification and Deep Learning for improved Answer Selection

Proceedings of the 27th International Conference on Computational Linguistics, pages 3283–3294 Santa Fe, New Mexico, USA, August 20-26, 2018. 3283 Integrating Question Classification and Deep Learning for improved Answer Selection Harish Tayyar Madabushi School of Computer Science, University of Birmingham, United Kingdom. H.T.Madabushi@gmail.com Mark Lee School of Computer Science, University of Birmingham, United Kingdom. M.G.Lee@cs.bham.ac.uk John Barnden School of Computer Science, University of Birmingham, United Kingdom. J.A.Barnden@cs.bham.ac.uk Abstract We present a system for Answer Selection that integrates fine-grained Question Classification with a Deep Learning model designed for Answer Selection. We detail the necessary changes to the Question Classification taxonomy and system, the creation of a new Entity Identification system and methods of highlighting entities to achieve this objective. Our experiments show that Question Classes are a strong signal to Deep Learning models for Answer Selection, and enable us to outperform the current state of the art in all variations of our experiments except one. In the best configuration, our MRR and MAP scores outperform the current state of the art by between 3 and 5 points on both versions of the TREC Answer Selection test set, a standard dataset for this task. 1 Introduction and Motivation Question Answering (QA) is the task of automatically generating answers to questions posed in natu- ral language. The task has received significant attention from researchers over several decades with a renewed interest in recent times. Current interest has been partly due to significant improvements in Natural Language Processing, and partly due to the increase in demand for such systems, amongst both the general populace, and corporate entities. An important subtask of QA is Question Classification (QC), which deals with the classification of questions based on the expected class of the answer. For example, the question “How much does the Capitol Dome weigh?” could be classified into the class “Numeric, Weight”, while the question “Name the actress in the movie Titanic” could be classified into the class “Human, Individual”. While there exist QA systems that do not make use of QC, the addition of QC to a QA system has been shown to improve its accuracy (Hovy et al., 2001). The specific classification that a QC system uses is called its taxonomy, and taxonomies vary quite widely in both specificity and form. Intuitively, QC improves QA by reducing the search space of potential answers, thus making the discovery of answers more efficient and accurate. While research into QC is fairly mature, fine grained QC is not always used for QA. For example, Tsai et al. (2015) use very simple rules based partly on question wh-words (i.e. “what”, “where”, “who”, etc.), resulting in 13 question classes. Not only is this number low compared to the 50 classes defined by (Li and Roth, 2002) or the over 120 used by (Hermjakob, 2001), the rules themselves are fairly weak. As an example, Tsai et al. (2015) classify all questions containing “name” under the class “Name” (“Name the country most famous for cheese.”) Most QA systems consist primarily of …

Content truncated to 500 words