Extending a Thesaurus with Words from Pan-Chinese Sources

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 457–464 Manchester, August 2008 Extending a Thesaurus with Words from Pan-Chinese Sources Oi Yee Kwong†‡ and Benjamin K. Tsou‡ †Department of Chinese, Translation and Linguistics ‡Language Information Sciences Research Centre City University of Hong Kong Tat Chee Avenue, Kowloon, Hong Kong {rlolivia, rlbtsou}@cityu.edu.hk Abstract In this paper, we work on extending a Chinese thesaurus with words distinctly used in various Chinese communities. The acquisition and classification of such region-specific lexical items is an impor- tant step toward the larger goal of con- structing a Pan-Chinese lexical resource. In particular, we extend a previous study in three respects: (1) to improve auto- matic classification by removing dupli- cated words from the thesaurus, (2) to experiment with classifying words at the subclass level and semantic head level, and (3) to further investigate the possible effects of data heterogeneity between the region-specific words and words in the thesaurus on classification performance. Automatic classification was based on the similarity between a target word and individual categories of words in the the- saurus, measured by the cosine function. Experiments were done on 120 target words from four regions. The automatic classification results were evaluated against a gold standard obtained from human judgements. In general accuracy reached 80% or more with the top 10 (out of 80+) and top 100 (out of 1,300+) can- didates considered at the subclass level and semantic head level respectively, provided that the appropriate data sources were used. © 2008. Licensed under the Creative Commons Attri- bution-Noncommercial-Share Alike 3.0 Unported license (http://creativecommons.org/licenses/by-nc- sa/3.0/). Some rights reserved. 1 Introduction A unique problem in Chinese language process- ing arises from the extensive lexical variations among major Chinese speech communities. Al- though different communities (e.g. Beijing, Hong Kong, Taipei and Singapore) often share a large core lexicon, lexical variations could occur in at least two ways. On the one hand, even the same word forms shared by various communities could be used with different meanings. For instance, the word 居屋 (ju1wu1)1 refers to general hous- ing in Mainland China but specifically to housing under the Home Ownership Scheme in Hong Kong. On the other hand, there are substantially different lexical items used for lexicalizing common or region-specific concepts. For exam- ple, while the word 住房 (zhu4fang2) is similarly used as 居 屋 to mean general housing in Mainland China, it is rarely seen in the Hong Kong context; and 下崗 (xia4gang3) is specific, if not exclusive, to Mainland China for referring to a special concept of unemployment. Existing Chinese lexical resources are often based on language use in one particular region and are therefore not comprehensive enough to capture the substantial regional variation as an important part of the lexical knowledge, which will be useful and critical for many NLP applica- tions, including natural language understanding, information retrieval, and machine translation. Tsou and Kwong (2006) proposed a compre- hensive Pan-Chinese lexical resource, using a large and unique synchronous Chinese corpus as an …

Content truncated to 500 words