SemEval-2007 Task 05: Multilingual Chinese-English Lexical Sample

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 19–23, Prague, June 2007. c⃝2007 Association for Computational Linguistics SemEval-2007 Task 5: Multilingual Chinese-English Lexical Sample Peng Jin, Yunfang Wu and Shiwen Yu Institute of Computational Linguistics Peking University, Beijing China {jandp, wuyf, yusw}@pku.edu.cn Abstract The Multilingual Chinese-English lexical sample task at SemEval-2007 provides a framework to evaluate Chinese word sense disambiguation and to promote research. This paper reports on the task preparation and the results of six participants. 1 Introduction The Multilingual Chinese-English lexical sample task is designed following the leading ideas of the Senseval-3 Multilingual English-Hindi lexical sample task (Chklovski et al., 2004). The “sense tags” for the ambiguous Chinese target words are given in the form of their English translations. The data preparation is introduced in the second section. And then the participating systems are briefly described and their scores are listed. In the conclusions we bring forward some sug- gestion for the next campaign. 2 Chinese Word Sense Annotated Corpus All the training and test data come from the People’s Daily in January, February and March of 2000. The People’s Daily is the most popular newspaper in China and is open domain. Before manually sense annotating, the texts have been word-segmented and part of speech (PoS) tagged according to the PoS tagging scheme of Institute of Computational Linguistics in Peking University (ICL/PKU). The corpus had been used as one of the gold-standard data set for the second international Chinese word segmentation bakeoff in 2005.1 2.1 Manual Annotation The sense annotated corpus is manually con- structed with the help of a word sense annotating interface developed in Java. Three native annota- tors, two major in Chinese linguistics and one ma- jor in computer science took part in the construc- tion of the sense-annotated corpus. A text generally is first annotated by one annotator and then veri- fied by two checkers. Checking is of course a nec- essary procedure to keep the consistency. Inspired by the observation that checking all the instances of a word in a specific time frame will greatly im- prove the precision and accelerate the speed, a software tool is designed in Java to gather all the occurrences of a word in the corpus into a check- ing file with the sense KWIC (Key Word in Con- text) format in sense tags order. The inter- annotator agreement gets to 84.8% according to Wu. et al. (2006). The sense entries are specified in the Chinese Semantic Dictionary (CSD) developed by ICL/PKU. The sense distinctions are made mainly according to the Contemporary Chinese Dictionary, the most widely used dictionary in mandarin Chi- nese, with necessary adjustment and improvement is implemented according to words usage in real texts. Word senses are described using the feature- based formalism. The features, which appear in the form “Attribute =Value”, can incorporate ex- tensive distributional information about a word sense. The feature set constitutes the representation of a sense, while the verbal definitions of meaning 1 http://sighan.cs.uchicago.edu/bakeoff2005/ 19 serve only …

Content truncated to 500 words