PKU: Combining Supervised Classifiers with Features Selection

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 261–263, Prague, June 2007. c⃝2007 Association for Computational Linguistics PKU: Combining Supervised Classifiers with Features Selection Peng Jin, Danqing Zhu, *Fuxin Li and Yunfang Wu Institute of Computational Linguistics Peking University, Beijing, China *Institute of Automation Chinese Academy of Sciences Beijing, China {jandp,zhudanqing,wuyf}@pku.edu.cn *Fuxin.li@ia.ac.cn Abstract This paper presents the word sense disam- biguation system of Peking University which was designed for the SemEval-2007 competition. The system participated in the Web track of task 11 “English Lexical Sample Task via English-Chinese Parallel Text”. The system is a hybrid model by combining two supervised learning algo- rithms SVM and ME. And the method of entropy-based feature chosen was experi- mented. We obtained precision (and recall) of 81.5%. 1 Introduction The PKU system participated in the web track of task 11. In this task, the organizers propose an English lexical sample task for word sense disam- biguation (WSD), where the sense-annotated ex- amples are (semi)-automatically gathered from word-aligned English-Chinese parallel texts. After assigning appropriate Chinese translations to each sense of an English word, the English side of the parallel texts can then serve as the training data, as they are considered to have been disambiguated and "sense-annotated" by the appropriate Chinese translations. This proposed task is thus similar to the multilingual lexical sample task in Senseval3, except that the training and test examples are col- lected without manually annotating each individual ambiguous word occurrence. The system consists of two supervised learning classifiers, support vector machines (SVM) and maximum entropy (ME). A method of entropy- based feature chosen was experimented to reduce the feature dimensions. The training data was lim- ited to the labeled data provided by the task, and a PoS-tagger (tree-tagger) was used to get more fea- tures. 2 Features Selection We used tree-tagger to PoS-tag the texts before the feature extractor. No other resource is used in the system. The window size of the context is set to 5 around the ambiguous word. Only the following features are used in the system: Local words Local PoSs Bag-of-words Local collocations Here local collocation means any two words which fall into the context window to form collo- cation pair. Two methods are used to reduce the dimensions of feature space. One comes from the linguistic knowledge, some words whose PoSs are IN, DT, SYM, POS, CC or “``” are not included as the fea- tures. The second method is based on entropy. To each word, the training data was split to two parts for parameter estimation. One (usually consist of 30 – 50 instances) as the simultaneous test and the rest instances form the other part. First the entropy of each feature was calculated. For example, the target word ‘work’, it has two senses and the dimensions of its feature space is N. For feature , if it appears in m instances belong- ing to sense A and n instances in sense B. So the if 261 probability distributions are: n mm …

Content truncated to 500 words