Learning Unsupervised SVM Classifier for Answer Selection in Web Question Answering

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 33–41, Prague, June 2007. c⃝2007 Association for Computational Linguistics Learning Unsupervised SVM Classifier for Answer Selection in Web Question Answering Youzheng Wu, Ruiqiang Zhang, Xinhui Hu, and Hideki Kashioka National Institute of Information and Communications Technology (NICT), ATR Spoken Language Communication Research Labs. 2-2-2 Hikaridai ”Keihanna Science City” Kyoto 619-0288 Japan {Youzheng.wu,Ruiqiang.zhang,Xinhui.hu,Hideki.kashioka}@atr.jp Abstract Previous machine learning techniques for answer selection in question answering (QA) have required question-answer train- ing pairs. It has been too expensive and labor-intensive, however, to collect these training pairs. This paper presents a novel unsupervised support vector machine (U- SVM) classifier for answer selection, which is independent of language and does not re- quire hand-tagged training pairs. The key ideas are the following: 1. unsupervised learning of training data for the classifier by clustering web search results; and 2. select- ing the correct answer from the candidates by classifying the question. The compara- tive experiments demonstrate that the pro- posed approach significantly outperforms the retrieval-based model (Retrieval-M), the supervised SVM classifier (S-SVM), and the pattern-based model (Pattern-M) for answer selection. Moreover, the cross-model com- parison showed that the performance rank- ing of these models was: U-SVM > Pattern- M > S-SVM > Retrieval-M. 1 Introduction The purpose of answer selection in QA is to se- lect the exact answer to the question from the ex- tracted candidate answers. In recent years, many supervised machine learning techniques for answer selection in open-domain question answering have been investigated in some pioneering studies [Itty- cheriah et al. 2001; Ng et al. 2001; Suzuki et al. 2002; Sasaki, et al. 2005; and Echihabi et al. 2003]. Compared with retrieval-based [Yang et al. 2003], pattern-based [Ravichandran et al. 2002 and Soub- botin et al. 2002], and deep NLP-based [Moldovan et al. 2002, Hovy et al. 2001; and Pasca et al. 2001] answer selection, machine learning techniques are more effective in constructing QA components from scratch. These techniques suffer, however, from the problem of requiring an adequate number of hand- tagged question-answer training pairs. It is too ex- pensive and labor intensive to collect such training pairs for supervised machine learning techniques. To tackle this knowledge acquisition bottleneck, this paper presents an unsupervised SVM classifier for answer selection, which is independent of lan- guage and question type, and avoids the need for hand-tagged question-answer pairs. The key ideas are as follows: 1. Regarding answer selection as a kind of classi- fication task and adopting an SVM classifier; 2. Applying unsupervised learning of pseudo- training data for the SVM classifier by cluster- ing web search results; 3. Training the SVM classifier by using three types of features extracted from the pseudo- training data; and 4. Selecting the correct answer from the candidate answers by classifying the question. Note that this means classifying a question into one of the clusters learned by clustering web search results. Therefore, our classifying the question 33 Figure 1: Web …

Content truncated to 500 words