Evaluating Question-Answering Techniques in Chinese

Evaluating Question-Answering Techniques in Chinese Xiaoyan Li and W. Bruce Croft Computer Science Department University of Massachusetts, Amherst, MA {xiaoyan, croft}@cs.umass.edu ABSTRACT An important first step in developing a cross-lingual question answering system is to understand whether techniques developed with English text will also work with other languages, such as Chinese. The Marsha Chinese question answering system described in this paper uses techniques similar to those used in the English systems developed for TREC. Marsha consists of three main components: the query processing module, the Hanquery search engine, and the answer extraction module. It also contains some specific techniques dealing with Chinese language characteristics, such as word segmentation and ordinals processing. Evaluation of the system is done using a method based on the TREC question-answering track. The results of the evaluation show that the performance of Marsha is comparable to some English question answering systems in TREC 8 track. An English language version of Marsha further indicates that the heuristics used are applicable to the English question answering task. Keywords Question-Answering (QA); Search engine; multilingual retrieval, Chinese QA. 1. Introduction A number of techniques for “question answering” have recently been evaluated both in the TREC environment (Voorhees and Harman, 1999) and in the DARPA TIDES program. In the standard approach to information retrieval, relevant text documents are retrieved in response to a query. The parts of those documents that may contain the most useful information or even the actual answer to the query are typically indicated by highlighting occurrences of query words in the text. In contrast, the task of a question- answering system is to identify text passages containing the relevant information and, if possible, extract the actual answer to the query. Question answering has a long history in natural language processing, and Salton’s first book (Salton, 1968) contains a detailed discussion of the relationship between information retrieval and question- answering systems. The focus in recent research has been on extracting answers from very large text databases and many of the techniques use search technology as a major component. A significant number of the queries used in information retrieval experiments are questions, for example, TREC topic 338 “What adverse effects have people experienced while taking aspirin repeatedly?” and topic 308 “What are the advantages and/or disadvantages of tooth implants?” In question-answering experiments, the queries tend to be more restricted questions, where answers are likely to be found in a single text passage, for example, TREC question-answering question 11 “Who was President Cleveland’s wife?” and question 14 “What country is the biggest producer of Tungsten?” The TREC question-answering experiments have, to date, used only English text. As the first step towards our goal of cross-lingual question answering, we investigated whether the general approaches to question answering that have been used in English will also be effective for Chinese. Although it is now well known that statistical information retrieval techniques are effective in many languages, earlier research, such as Fujii and Croft (1993, 1999), was helpful in pointing out which techniques …

Content truncated to 500 words