WSD2: Parameter optimisation for Memory-based Cross-Lingual Word-Sense Disambiguation

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 183–187, Atlanta, Georgia, June 14-15, 2013. c⃝2013 Association for Computational Linguistics WSD2: Parameter optimisation for Memory-based Cross-Lingual Word-Sense Disambiguation Maarten van Gompel and Antal van den Bosch Centre for Language Studies, Radboud University Nijmegen proycon@anaproy.nl, a.vandenbosch@let.ru.nl Abstract We present our system WSD2 which partic- ipated in the Cross-Lingual Word-Sense Dis- ambiguation task for SemEval 2013 (Lefever and Hoste, 2013). The system closely resem- bles our winning system for the same task in SemEval 2010. It is based on k-nearest neigh- bour classifiers which map words with local and global context features onto their transla- tion, i.e. their cross-lingual sense. The sys- tem participated in the task for all five lan- guages and obtained winning scores for four of them when asked to predict the best trans- lation(s). We tested various configurations of our system, focusing on various levels of hy- perparameter optimisation and feature selec- tion. Our final results indicate that hyperpa- rameter optimisation did not lead to the best results, indicating overfitting by our optimisa- tion method in this aspect. Feature selection does have a modest positive impact. 1 Introduction WSD2 is a rewrite and extension of our previous system (van Gompel, 2010) that participated in the Cross-Lingual Word Sense Disambiguation task in SemEval 2010 (Lefever and Hoste, 2010). In WSD2 we introduce and test a new level of hyperparame- ter optimisation. Unlike the previous occasion, we participate in all five target languages (Dutch, Span- ish, Italian, French, and German). The task presents twenty polysemous nouns with fifty instances each to be mapped onto normalised (lemmatised) transla- tions in all languages. The task is described in detail by Lefever and Hoste (2013). Trial data is provided and has been used to op- timise system parameters. Due to the unsupervised nature of the task, no training data is provided. How- ever, given that the gold standard of the task is based exclusively on the Europarl parallel corpus (Koehn, 2005), we select that same corpus to minimise our chances of delivering translations that the human annotators preparing the test data could have never picked. Systems may output several senses per instance, rather than producing just one sense prediction. These are evaluated in two different ways. The scor- ing type “best” expects that the system outputs the sense it considers the most likely, or a number of senses in the order of its confidence in these senses being correct. Multiple guesses are penalised, how- ever. In contrast, the scoring type “out of five” ex- pects five guesses, in which each answer carries the same weight. These metrics are more extensively described in Mihalcea et al. (2010) and Lefever and Hoste (2013). 2 System Description The WSD2 system, like its predecessor, distributes the task over word experts. Each word expert is a k-nearest neighbour classifier specialising in the disambiguation of a single of the twenty pro- vided nouns. This is implemented using the …

Content truncated to 500 words