Categorial Fluidity in Chinese and its Implications for Part-of-speech Tagging

Categorial Fluidity in Chinese and its Implications for Part-of-speech Tagging OiYeeKwong� Benjamin K. Tsou Language Information Sciences Research Centre City University of Hong Kong, Kowloon, Hong Kong {rlolivia, rlbtsou}@cityu.edu.hk Abstract This paper discusses the theoretical and practical concerns in part-of-speech (POS) tagging for Chinese. Unlike other languages such as English, Chinese lacks morphological marking in association with categorial alternations. We consider such categorial fluidity a continuum, and any categorial shift a transition, with spe- cial focus on the verb-noun shift. Pre- liminary observations are reported on this phenomenon from empirical data, and we suggest that POS tagging should not only be theoretically valid but also sufficiently capture the extent of categorial fluidity as reflected by the data. 1 Introduction There are currently a number of POS-tagged Chinese corpora available, based on different tagsets and theoretical frameworks. Some are more semantic-oriented in determining the syn- tactic category of a word. For instance, CKIP (1993) had a very fine-grained classification of Chinese verbs based on thematic structures. Oth- ers are mainly based on syntactic distribution (e.g. Yu et al., 1998; Xia, 2000) where POS tags are assigned mainly depending on the syntactic properties of the target words. Given the many-to-many relation between grammatical function and lexical category in Chinese, it is often not straightforward as to how certain words should be tagged in particular sen- tences. For example, should the verb Ilk huai2yi2 (to suspect) be given the same tag in all cases in (1) 1 ? The digits following the Hanyu pinyin indicate the tone. (1) a. f-v-mt 'IBA (I suspect he is a thief) wo3 huai2yi2 to] shi4 zei2 b. ftiORMV*M (He wears a suspicious look) to] man3han3 huai2yi2 biao3qing2 c. Rnfilm (This is only my suspicion) zhe4 zhi3shi4 vvo3 de5 huai2yi2 Most Chinese grammarians would suggest that Chinese words have predefined lexical categories mainly based on their syntactic properties, which should not be contingent upon the grammatical function the words play in particular sentences (e.g. Zhu, 2001). In this study, however, we will show that categorial fluidity (i.e. the relative flexibility of a word being used for different grammatical functions) should be captured in the tagging of large corpora to provide an important resource for the study of this special linguistic phenomenon, as well as for lexicography and natural language processing. In Section 2, we first discuss POS ambiguity and categorial fluidity in Chinese and the diffi- culty thus posed on tagging. Then in Section 3, we report on a preliminary empirical study of the categorial fluidity and shift between Chinese verbs and nouns. The implications for POS tag- ging are explored in Section 4, followed by a conclusion with future work in Section 5. 2 POS Ambiguity in Chinese 2.1 The Difficulty The major problem in POS tagging, for all lan- guages alike, is probably that of ambiguity. Where a word has multiple potential POS tags, the ambiguity has to be resolved in context. The problem of POS ambiguity is especially salient for Chinese, mainly for two reasons. …

Content truncated to 500 words