RACAI: Unsupervised WSD Experiments @ SemEval-2, Task 17

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 411–416, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics RACAI: Unsupervised WSD experiments @ SemEval-2, Task #17 Radu Ion Institute for AI, Romanian Academy 13, Calea 13 Septembrie, Bucharest 050711, Romania radu@racai.ro Dan Ştefănescu Institute for AI, Romanian Academy 13, Calea 13 Septembrie, Bucharest 050711, Romania danstef@racai.ro Abstract This paper documents the participation of the Research Institute for Artificial Intelligence of the Romanian Academy (RACAI) to the Task 17 – All-words Word Sense Disambiguation on a Specific Domain, of the SemEval-2 com- petition. We describe three unsupervised WSD systems that make extensive use of the Prince- ton WordNet (WN) structure and WordNet Domains in order to perform the disambigua- tion. The best of them has been ranked the 12th by the task organizers out of 29 judged runs. 1 Introduction Referring to the last SemEval (SemEval-1, (Agirre et al., 2007a)) and to our recent work (Ion and Ştefănescu, 2009), unsupervised Word Sense Disambiguation (WSD) is still at the bot- tom of WSD systems ranking with a significant loss in performance when compared to super- vised approaches. With Task #17 @ SemEval-2, this observation is (probably 1 ) reinforced but another issue is re-brought to light: the difficulty of supervised WSD systems to adapt to a given domain (Agirre et al., 2009). With general scores lower with at least 3% than 3 years ago in Task #17 @ SemEval-1 which was a supposedly hard- er task (general, no particular domain WSD was required for all words), we observe that super- vised WSD is certainly more difficult to imple- ment in a real world application. Our unsupervised WSD approach benefited from the specification of this year’s Task #17 which was a domain-limited WSD, meaning that the disambiguation would be applied to content words drawn from a specific domain: the sur- rounding environment. We worked under the assumption that a term of the given domain 1 At the time of the writing we only know the systems rank- ing without the supervised/unsupervised distinction. would have the same meaning with all its occur- rences throughout the text. This hypothesis has been put forth by Yarowsky (1993) as the “one sense per discourse” hypothesis (OSPD for short). The task organizers offered a set of back- ground documents with no sense annotations to the competitors who want to train/tune their sys- tems using data from the same domain as the official test set. Working with the OSPD hypo- thesis, we set off to construct/test domain specif- ic WSD models from/on this corpus using the WordNet Domains (Bentivogli et al., 2004). For testing purposes, we have constructed an in- house gold standard from this corpus that com- prises of 1601 occurrences of 204 terms of the “surrounding environment” domain that have been automatically extracted with the highest confidence. We have observed that our gold standard (which has been independently anno- tated by 3 annotators but on non-overlapping sections which led to having no …

Content truncated to 500 words