Investigating the Relationship between Word Segmentation Performance and Retrieval Performance in Chinese IR

Investigating the Relationship between Word Segmentation Performance and Retrieval Performance in Chinese IR Fuchun Peng and Xiangji Huang and Dale Schuurmans and Nick Cercone School of Computer Science, University of Waterloo 200 University Ave. West, Waterloo, Ontario, Canada, N2L 3G1 {f3peng, jhuang, dale, ncercone}@uwaterloo.ca Abstract It is commonly believed that word segmentation ac- curacy is monotonically related to retrieval perfor- mance in Chinese information retrieval. In this pa- per we show that, for Chinese, the relationship be- tween segmentation and retrieval performance is in fact nonmonotonic; that is, at around 70% word segmentation accuracy an over-segmentation phe- nomenon begins to occur which leads to a reduction in information retrieval performance. We demon- strate this effect by presenting an empirical inves- tigation of information retrieval on Chinese TREC data, using a wide variety of word segmentation al- gorithms with word segmentation accuracies ranging from 44% to 95%. It appears that the main reason for the drop in retrieval performance is that correct compounds and collocations are preserved by accu- rate segmenters, while they are broken up by less accurate (but reasonable) segmenters, to a surpris- ing advantage. This suggests that words themselves might be too broad a notion to conveniently capture the general semantic meaning of Chinese text. 1 Introduction Automated processing of written languages such as Chinese involves an inherent word segmentation problem that is not present in western languages like English. Unlike English, Chinese words are not ex- plicitly delimited by whitespace, and therefore to perform automated text processing tasks (such as information retrieval) one normally has to first seg- ment the text collection. Typically this involves seg- menting the text into individual words. Although the text segmentation problem in Chinese has been heavily investigated recently (Brent and Tao, 2001; Chang, 1997; Ge et al., 1999; Hockenmaier and Brew, 1998; Jin, 1992; Peng and Schuurmans, 2001; Sproat and Shih, 1990; Teahan et al, 2001) most research has focused on the problem of segmenting character strings into individual words, rather than useful constituents. However, we have found that focusing exclusively on words may not lead to the most effective segmentation from the perspective of broad semantic analysis (Peng et al, 2002). In this paper we will focus on a simple form of se- mantic text processing: information retrieval (IR). Although information retrieval does not require a deep semantic analysis, to perform effective retrieval one still has to accurately capture the main topic of discourse and relate this to a given query. In the con- text of Chinese, information retrieval is complicated by the fact that the words in the source text (and perhaps even the query) are not separated by whites- pace. This creates a significant amount of additional ambiguity in interpreting sentences and identifying the underlying topic of discourse. There are two standard approaches to information retrieval in Chinese text: character based and word based. It is usually thought that word based ap- proaches should be superior, even though character based methods are simpler and more commonly used (Huang …

Content truncated to 500 words