Value for Money: Balancing Annotation Effort, Lexicon Building and Accuracy for Multilingual WSD

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 555–563, Beijing, August 2010 Value for Money: Balancing Annotation Effort, Lexicon Building and Accuracy for Multilingual WSD Mitesh M. Khapra Saurabh Sohoney Anup Kulkarni Pushpak Bhattacharyya Department of Computer Science and Engineering, Indian Institute of Technology Bombay {miteshk,saurabhsohoney,anup,pb}@cse.iitb.ac.in Abstract Sense annotation and lexicon building are costly affairs demanding prudent invest- ment of resources. Recent work on mul- tilingual WSD has shown that it is possi- ble to leverage the annotation work done for WSD of one language (SL) for another (TL), by projecting Wordnet and sense marked corpus parameters of SL to TL. However, this work does not take into ac- count the cost of manually cross-linking the words within aligned synsets. Further, it does not answer the question of “Can better accuracy be achieved if a user is willing to pay additional money?” We propose a measure for cost-benefit analy- sis which measures the “value for money” earned in terms of accuracy by invest- ing in annotation effort and lexicon build- ing. Two key ideas explored in this pa- per are (i) the use of probabilistic cross- linking model to reduce manual cross- linking effort and (ii) the use of selective sampling to inject a few training examples for hard-to-disambiguate words from the target language to boost the accuracy. 1 Introduction Word Sense Disambiguation (WSD) is one of the most widely investigated problems of Natural Language Processing (NLP). Previous works have shown that supervised approaches to Word Sense Disambiguation which rely on sense annotated corpora (Ng and Lee, 1996; Lee et al., 2004) out- perform unsupervised (Veronis, 2004) and knowl- edge based approaches (Mihalcea, 2005). How- ever, creation of sense marked corpora has always remained a costly proposition, especially for some of the resource deprived languages. To circumvent this problem, Khapra et al. (2009) proposed a WSD method that can be ap- plied to a language even when no sense tagged corpus for that language is available. This is achieved by projecting Wordnet and corpus pa- rameters from another language to the language in question. The approach is centered on a novel synset based multilingual dictionary (Mohanty et al., 2008) where the synsets of different languages are aligned and thereafter the words within the synsets are manually cross-linked. For example, the word WL1 belonging to synset S of language L1 will be manually cross-linked to the word WL2 of the corresponding synset in language L2 to in- dicate that WL2 is the best substitute for WL1 ac- cording to an experienced bilingual speaker’s in- tuition. We extend their work by addressing the follow- ing question on the economics of annotation, lex- icon building and performance: • Is there an optimal point of balance between the annotation effort and the lexicon build- ing (i.e. manual cross-linking) effort at which one can be assured of best value for money in terms of accuracy? To address the above question we first propose a probabilistic cross linking model to eliminate the …

Content truncated to 500 words