Using the Web as Corpus for Un-supervised Learning in Question Answering

Using the Web as Corpus for Un-supervised Learning in Question Answering Yi-Chia Wang1, Jian-Cheng Wu2, Tyne Liang1 and Jason S. Chang2 1. Dep. of Computer and Information Science, National Chiao Tung University, Taiwan, R.O.C. 2. Dep. of Computer Science, National Tsing Hua University, Taiwan, R.O.C. rhyme.cis92g@nctu.edu.tw Abstract. In this paper we propose a method for unsupervised learning of relation between terms in questions and answer passages by using the Web as corpus. The method involves automatic acquisition of relevant answer passages from the Web for a set of questions and answers, as well as alignment of wh -phrases and keywords in questions with phrases in the answer passages. At run time, wh-phrases and keywords are transformed to a sequence of expanded query terms in order to bias the underlying search engine to give higher rank to relevant passages. Evaluation on a set of questions shows that our prototype improves the performance of a question answering system by increasing the precision rate of top ranking passages returned by the search engine. 1. Introduction It was noted that people have submitted longer and longer queries to the Web search engines. Recently, users have started to submit natural language queries instead of a list of keywords. It has encouraged many researchers to develop question answering systems which specifically aim at natural language questions, such as AskJeeves (www.ask.com) and START (www.ai.mit.edu/projects/infolab/). For typical question answering systems, document/passage retrieval is the most significant subtask. In this step, the QA system breaks a natural language question into a set of keywords, uses keywords to query a search engine, and returns documents or messages that are related to the queries for further processing. However, the keywords in questions usually are not very effective in retrieving relevant passages. Consider the question “Who invented glasses with two foci?” Typically, we will send the keywords “invented glasses two foci” to a search engine to retrieve documents or passages. Submitting such keywords to AltaVista, we got irrelevant information about astronomy or physics rather than the inventor “Benjamin Franklin” of bifocal glasses. Intuitively, if we include the phrase “inventor of” or “bifocal” in the query sent to the search engine (SE), we are likely to retrieve passages with the answer. We present the system Atlas (Automatic Transform Learning by Aligning Sentences of question and answer), which automatically learns the transforms from wh -phrases and keywords to n-grams in relevant passages by using the Web as corpus. The transformed query should be more likely to retrieve passages that contain the answer. For instance, consider the natural language question “Who invented the light bulb?” Using the keywords in the question directly, we end up with the keyword query, “invented light bulb,” for a search engine such as Google. We observed that such a query has room for improvement in terms of bringing in more instances of the relevant answer. Our experiment indicates that the proposed method will determine the best transforms for the wh-phrase “who invented” including “inventor of”, “was invented”, and “invented by”. …

Content truncated to 500 words