Investigation of Question Classifier in Question Answering

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 543–550, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Investigation of Question Classifier in Question Answering Zhiheng Huang EECS Department University of California at Berkeley CA 94720-1776, USA zhiheng@cs.berkeley.edu Marcus Thint Intelligent Systems Research Center British Telecom Group Chief Technology Office marcus.2.thint@bt.com Asli Celikyilmaz EECS Department University of California at Berkeley CA 94720-1776, USA asli@cs.berkeley.edu Abstract In this paper, we investigate how an ac- curate question classifier contributes to a question answering system. We first present a Maximum Entropy (ME) based question classifier which makes use of head word features and their WordNet hy- pernyms. We show that our question clas- sifier can achieve the state of the art per- formance in the standard UIUC question dataset. We then investigate quantitatively the contribution of this question classifier to a feature driven question answering sys- tem. With our accurate question classifier and some standard question answer fea- tures, our question answering system per- forms close to the state of the art using TREC corpus. 1 Introduction Question answering has drawn significant atten- tion from the last decade (Prager, 2006). It at- tempts to answer the question posed in natural language by providing the answer phrase rather than the whole documents. An important step in question answering (QA) is to classify the ques- tion to the anticipated type of the answer. For example, the question of Who discovered x-rays should be classified into the type of human (indi- vidual). This information would narrow down the search space to identify the correct answer string. In addition, this information can suggest different strategies to search and verify a candidate answer. In fact, the combination of question classification and the named entity recognition is a key approach in modern question answering systems (Voorhees and Dang, 2005). ial:The question classification is by no means triv- Simply using question wh-words can not achieve satisfactory results. The difficulty lies in classifying the what and which type questions. Considering the example What is the capital of Yu- goslavia, it is of location (city) type, while What is the pH scale is of definition type. As with the previous work of (Li and Roth, 2002; Li and Roth, 2006; Krishnan et al., 2005; Moschitti et al., 2007), we propose a feature driven statistical question classifier (Huang et al., 2008). In partic- ular, we propose head word feature and augment semantic features of such head words using Word- Net. In addition, Lesk’s word sense disambigua- tion (WSD) algorithm is adapted and the depth of hypernym feature is optimized. With further aug- ment of other standard features such as unigrams, we can obtain accuracy of 89.0% using ME model for 50 fine classes over UIUC dataset. In addition to building an accurate question classifier, we investigate the contribution of this question classifier to a feature driven question an- swering rank model. It is worth noting that, most of the features we used in question answering rank model, depend on …

Content truncated to 500 words