High Accuracy Rule-based Question Classification using Question Syntax and Semantics

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1220–1230, Osaka, Japan, December 11-17 2016. High Accuracy Rule-based Question Classification using Question Syntax and Semantics Harish Tayyar Madabushi School of Computer Science, University of Birmingham, Birmingham, United Kingdom. H.T.Madabushi@cs.bham.ac.uk Mark Lee School of Computer Science, University of Birmingham, Birmingham, United Kingdom. M.G.Lee@cs.bham.ac.uk Abstract We present in this paper a purely rule-based system for Question Classification which we divide into two parts: The first is the extraction of relevant words from a question by use of its structure, and the second is the classification of questions based on rules that associate these words to Concepts. We achieve an accuracy of 97.2%, close to a 6 point improvement over the previous State of the Art of 91.6%. Additionally, we believe that machine learning algorithms can be applied on top of this method to further improve accuracy. 1 Introduction and Motivation Question Answering (QA) is a task in Natural Language Processing (NLP) that requires the system to provide concise answers to Natural Language questions. Interest in QA has grown dramatically over the past couple of years, in part due to advances in NLP and Machine Learning, that have allowed for significant improvements in QA systems, and in part due to its increased accessibility to the general public via smart-phone applications such as Siri and Google Now. An important element of QA is Question Classification (QC), which is the task of classifying a question based on the expected answer. As an example, the question “Who is the prime minister?” could be assigned the class “person”, whereas the question “Where is the prime minister?” could belong to the class “location”. Since the task involves identifying the type of answer, it is sometimes referred to as Answer Type Classification. While there do exist QA Systems that do not make use of QC, QC has been shown to significantly improve the performance of QA systems (Hovy et al., 2001). A priori knowledge of the kind of information that a QA system is required to extract allows for the exploitation of predefined patterns and improved feature selection. For example, consider a QA system provided with the information that the question “How long is the term of office of the Prime Minister?” requires, as an answer, a “number that represents a duration”. Such a system could dramatically reduce its search space, in that it could focus on numbers. The design of a QA system’s search and information extraction components determines the classes that a QC should use. Despite this dependence on how question classes are used, there are some common question classes that are widely accepted as useful. The rules that govern and classes contained in a given question classification are based on the specific Question Taxonomy chosen. 2 Related Work Work on QC, as in most NLP tasks, can be broadly divided into three categories: a) those that make use of machine learning, b) those that rely purely on rules, and c) those …

Content truncated to 500 words