Web-Based Unsupervised Learning for Query Formulation in Question Answering

R. Dale et al. (Eds.): IJCNLP 2005, LNAI 3651, pp. 519 – 529, 2005. © Springer-Verlag Berlin Heidelberg 2005 Web-Based Unsupervised Learning for Query Formulation in Question Answering Yi-Chia Wang1, Jian-Cheng Wu2, Tyne Liang1, and Jason S. Chang2 1 Dep. of Computer and Information Science, National Chiao Tung University, 1001 Ta Hsueh Rd., Hsinchu, Taiwan 300, R.O.C. rhyme.cis92g@nctu.edu.tw, tliang@cis.nctu.edu.tw 2 Dep. of Computer Science, National Tsing Hua University, 101, Section 2 Kuang Fu Road, Hsinchu, Taiwan 300, R.O.C. d928322@oz.nthu.edu.tw, jschang@cs.nthu.edu.tw Abstract. Converting questions to effective queries is crucial to open-domain question answering systems. In this paper, we present a web-based unsupervised learning approach for transforming a given natural-language question to an effective query. The method involves querying a search engine for Web passages that contain the answer to the question, extracting patterns that characterize fine-grained classification for answers, and linking these patterns with n-grams in answer passages. Independent evaluation on a set of questions shows that the proposed approach outperforms a naive keyword- based approach in terms of mean reciprocal rank and human effort. 1 Introduction An automated question answering (QA) system receives a user’s natural-language question and returns exact answers by analyzing the question and consulting a large text collection [1, 2]. As Moldovan et al. [3] pointed out, over 60% of the QA errors can be attributed to ineffective question processing, including query formulation and query expansion. A naive solution to query formulation is using the keywords in an input question as the query to a search engine. However, it is possible that the keywords may not appear in those answer passages which contain answers to the given question. For example, submitting the keywords in “Who invented washing machine?” to a search engine like Google may not lead to retrieval of answer passages like “The inventor of the automatic washer was John Chamberlain.” In fact, by expanding the keyword set (“invented”, “washing”, “machine”) with “inventor of,” the query to a search engine is effective in retrieving such answer passages as the top-ranking pages. Hence, if we can learn how to associate a set of questions (e.g. (“who invented …?”) with effective keywords or phrases (e.g. “inventor of”) which are likely to appear in answer passages, the search engine will have a better chance of retrieving pages containing the answer. In this paper, we present a novel Web-based unsupervised learning approach to handling question analysis for QA systems. In our approach, training-data questions are first analyzed and classified into a set of fine-grained categories of question 520 Y.-C. Wang et al. patterns. Then, the relationships between the question patterns and n-grams in answer passages are discovered by employing a word alignment technique. Finally, the best query transforms are derived by ranking the n-grams which are associated with a specific question pattern. At runtime, the keywords in a given question are extracted and the question is categorized. Then the keywords are expanded according the category of the question. The expanded query is the submitted to a search engine in order …

Content truncated to 500 words