The Senseval-3 Multilingual English-Hindi lexical sample task

The SENSEVAL–3 Multilingual English–Hindi Lexical Sample Task Timothy Chklovski Information Sciences Institute University of Southern California Marina del Rey, CA 90292 timc@isi.edu Rada Mihalcea Department of Computer Science University of North Texas Dallas, TX 76203 rada@cs.unt.edu Ted Pedersen Department of Computer Science University of Minnesota Duluth, MN 55812 tpederse@d.umn.edu Amruta Purandare Department of Computer Science University of Minnesota Duluth, MN 55812 pura0010@d.umn.edu Abstract This paper describes the English–Hindi Multilingual lexical sample task in SENSEVAL–3. Rather than tagging an English word with a sense from an En- glish dictionary, this task seeks to assign the most appropriate Hindi translation to an ambiguous tar- get word. Training data was solicited via the Open Mind Word Expert (OMWE) from Web users who are fluent in English and Hindi. 1 Introduction The goal of the MultiLingual lexical sample task is to create a framework for the evaluation of sys- tems that perform Machine Translation, with a fo- cus on the translation of ambiguous words. The task is very similar to the lexical sample task, ex- cept that rather than using the sense inventory from a dictionary we follow the suggestion of (Resnik and Yarowsky, 1999) and use the translations of the tar- get words into a second language. In this task for SENSEVAL-3, the contexts are in English, and the “sense tags” for the English target words are their translations in Hindi. This paper outlines some of the major issues that arose in the creation of this task, and then describes the participating systems and summarizes their re- sults. 2 Open Mind Word Expert The annotated corpus required for this task was built using the Open Mind Word Expert system (Chklovski and Mihalcea, 2002), adapted for mul- tilingual annotations 1. To overcome the current lack of tagged data and the limitations imposed by the creation of such data using trained lexicographers, the Open Mind Word 1Multilingual Open Mind Word Expert can be accessed at http://teach-computers.org/word-expert/english-hindi Expert system enables the collection of semantically annotated corpora over the Web. Tagged examples are collected using a Web-based application that al- lows contributors to annotate words with their mean- ings. The tagging exercise proceeds as follows. For each target word the system extracts a set of sen- tences from a large textual corpus. These examples are presented to the contributors, together with all possible translations for the given target word. Users are asked to select the most appropriate translation for the target word in each sentence. The selection is made using check-boxes, which list all possible translations, plus two additional choices, “unclear” and “none of the above.” Although users are encour- aged to select only one translation per word, the se- lection of two or more translations is also possible. The results of the classification submitted by other users are not presented to avoid artificial biases. 3 Sense Inventory Representation The sense inventory used in this task is the set of Hindi translations associated with the English words in our lexical sample. Selecting an appropriate English-Hindi dictionary was …

Content truncated to 500 words