An Empirical Study of Combing Multiple Models in Bengali Question Classification

International Joint Conference on Natural Language Processing, pages 892–896, Nagoya, Japan, 14-18 October 2013. An Empirical Study of Combining Multiple Models in Bengali Question Classification Somnath Banerjee Department of Computer Science and Engineering Jadavpur University, India s.banerjee1980@gmail.com Sivaji Bandyopadhyay Department of Computer Science and Engineering Jadavpur University, India sivaji cse ju@yahoo.com Abstract This paper demonstrates that combination of multiple models achieves better clas- sification performance than that obtained by existing individual models for ques- tion classification task in Bengali. We have exploited state of the art multiple model combination techniques, i.e., en- semble, stacking and voting on lexical, syntactical and semantic features of Ben- gali question for the question classification task. Bagging and boosting have been ex- perimented as ensemble techniques. Na¨ıve Bayes, kernel Na¨ıve Bayes, Rule Induc- tion and Decision Tree classifiers have been used as base learners. The exper- imental results show that classifier com- bination models outperform existing sin- gle model approaches. Overall voting approach has achieved maximum clas- sification accuracy of 91.65% and out- performed the existing single model ap- proaches (maximum accuracy of 87.63%). 1 Introduction Although different types of question answering systems (QA) have different architectures, most of them follow a framework in which question clas- sification (QC) plays an important role (Voorhees, 2001) and QC has significant influence on the overall performance of a QA system (Ittycheriah et al., 2001; Hovy et al., 2001; Moldovan et al., 2003). The task of a question classifier is to assign one or more class labels, depending on classifica- tion strategy, to a given question written in natural language. Basically there are two main motivations for question classification: locating the answer and choosing the search strategy. Knowing the ques- tion class not only reduces the search space needed to find the answer, it can also help to find the true answer in a given set of candidate answers. One of the main issues of classification model- ing is the improvement of classification accuracy. For that purpose, many researchers have recently placed considerable attention to the task of clas- sifier combination methods. The idea is not to rely on a single decision making scheme. Instead, many single classifiers are used for decision mak- ing by combining their individual opinions to ar- rive at a consensus decision. 2 Related Work and Motivations A lot of researches on QC, question taxonomies, and question features are being published con- tinuously. There are basically two different ap- proaches used to classify questions- one is rule based (Hull, 1999; Prager et al., 1999) and another is machine learning based (Zhang et al., 2003; Li and Roth, 2004). However, a number of re- searchers have also used some hybrid approaches which combine rule-based and machine learning based approaches (Huang et al., 2008; Silva et al., 2011). Many researchers have investigated the tech- nique of combining the predictions of multiple classifiers to produce a single classifier (Breiman, 1996; Clemen, 1989; Perrone, 1993; Wolpert, 1992). The resulting classifier is generally more accurate than any of the …

Content truncated to 500 words