Classifying Questions in Question Answering System Using Finite State Machines with a Simple Learning Approach

Classifying Questions in Question Answering System Using Finite State Machines with a Simple Learning Approach Mohammad Moinul Hoque University of Evora, Evora, Portugal moincse@yahoo.com Teresa Goncalves University of Evora, Evora, Portugal tcg@uevora.pt Paulo Quaresma L2F/INESC-ID & University of Évora, Portugal pq@uevora.pt Abstract Question Classification plays a significant part in Question Answering system. In order to obtain a classifier, we present in this pa- per1 a pragmatic approach that utilizes sim- ple sentence structures observed and learned from the question sentence patterns, trains a set of Finite State Machines (FSM) based on keywords appearing in the sentences and uses the trained FSMs to classify various questions to their relevant classes. Although, questions can be placed using various syntac- tic structures and keywords, we have careful- ly observed that this variation is within a small finite limit and can be traced down us- ing a limited number of FSMs and a simple semantic understanding instead of using complex semantic analysis. WordNet seman- tic meaning of various keywords to extend the FSMs capability to accept a wide variety of wording used in the questions. Various kinds of questions written in English lan- guage and belonging to diverse classes from the Conference and Labs of the Evaluation Forum’s Question Answering track are used for the training purpose and a separate set of questions from the same track is used for analyzing the FSMs competence to map the questions to one of the recognizable classes. With the use of learning strategies and appli- cation of simple voting functions along with training the weights for the keywords appear- ing in the questions, we have managed to achieve a classification accuracy as high as 94%. The system was trained by placing questions in various orders to see if the sys- tem built up from those orders have any sub- tle impact on the accuracy rate. The usability of this approach lies in its simplicity and yet it performs well to cope up with various sen- tence patterns. 1 This work was supported by national funds through FCT – Fundação para a Ciência e a Tecnologia, under project PEst- OE/EEI/LA0021/2013 1 Introduction Classifying a question to its appropriate class in an important subtask and plays a substantial role in the Question Answering (QA) systems. It can provide some useful clues for identifying poten- tial answers in large collections of texts. The goal of this current work is to develop a classifier us- ing Finite State Machines (FSM) to classify a set of questions into their relevant classes. Various techniques have already been tried by the com- munity either to classify a question to its relevant class or to a finer subclass of a specific class. Results of the error analysis acquired from an open domain QA system demonstrates that more or less 36.4% of the errors were generated due to the wrong classification of questions (Moldovan et al., 2003). So, this issue can be highlighted as a subject of interest and has arisen the aim of developing more …

Content truncated to 500 words