Extending a Thesaurus in the Pan-Chinese Context

Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 325–333, Prague, June 2007. c⃝2007 Association for Computational Linguistics Extending a Thesaurus in the Pan-Chinese Context Oi Yee Kwong and Benjamin K. Tsou Language Information Sciences Research Centre City University of Hong Kong Tat Chee Avenue, Kowloon, Hong Kong {rlolivia,rlbtsou}@cityu.edu.hk Abstract In this paper, we address a unique problem in Chinese language processing and report on our study on extending a Chinese the- saurus with region-specific words, mostly from the financial domain, from various Chinese speech communities. With the larger goal of automatically constructing a Pan-Chinese lexical resource, this work aims at taking an existing semantic classi- ficatory structure as leverage and incorpo- rating new words into it. In particular, it is important to see if the classification could accommodate new words from heterogene- ous data sources, and whether simple simi- larity measures and clustering methods could cope with such variation. We use the cosine function for similarity and test it on automatically classifying 120 target words from four regions, using different datasets for the extraction of feature vectors. The automatic classification results were evalu- ated against human judgement, and the per- formance was encouraging, with accuracy reaching over 85% in some cases. Thus while human judgement is not straightfor- ward and it is difficult to create a Pan- Chinese lexicon manually, it is observed that combining simple clustering methods with the appropriate data sources appears to be a promising approach toward its automatic construction. 1 Introduction Large-scale semantic lexicons are important re- sources for many natural language processing (NLP) tasks. For a significant world language such as Chinese, it is especially critical to capture the substantial regional variation as an important part of the lexical knowledge, which will be useful for many NLP applications, including natural lan- guage understanding, information retrieval, and machine translation. Existing Chinese lexical re- sources, however, are often based on language use in one particular region and thus lack the desired comprehensiveness. Toward this end, Tsou and Kwong (2006) pro- posed a comprehensive Pan-Chinese lexical re- source, based on a large and unique synchronous Chinese corpus as an authentic source for lexical acquisition and analysis across various Chinese speech communities. To allow maximum versatil- ity and portability, it is expected to document the core and universal substances of the language on the one hand, and also the more subtle variations found in different communities on the other. Dif- ferent Chinese speech communities might share lexical items in the same form but with different meanings. For instance, the word 居屋 refers to general housing in Mainland China but specifically to housing under the Home Ownership Scheme in Hong Kong; and while the word 住房 is similar to 居屋 to mean general housing in Mainland China, it is rarely seen in the Hong Kong context. Hence, the current study aims at taking an exist- ing Chinese thesaurus, namely the Tongyici Cilin 同義詞詞林, as leverage and extending it with …

Content truncated to 500 words