SemEval-2007 Task 11: English Lexical Sample Task via English-Chinese Parallel Text

Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 54–58, Prague, June 2007. c⃝2007 Association for Computational Linguistics SemEval-2007 Task 11: English Lexical Sample Task via English-Chinese Parallel Text Hwee Tou Ng and Yee Seng Chan Department of Computer Science National University of Singapore 3 Science Drive 2, Singapore 117543 {nght, chanys}@comp.nus.edu.sg Abstract We made use of parallel texts to gather train- ing and test examples for the English lexi- cal sample task. Two tracks were organized for our task. The first track used examples gathered from an LDC corpus, while the second track used examples gathered from a Web corpus. In this paper, we describe the process of gathering examples from the parallel corpora, the differences with similar tasks in previous SENSEVAL evaluations, and present the results of participating sys- tems. 1 Introduction As part of the SemEval-2007 evaluation exercise, we organized an English lexical sample task for word sense disambiguation (WSD), where the sense- annotated examples were semi-automatically gath- ered from word-aligned English-Chinese parallel texts. Two tracks were organized for this task, each gathering data from a different corpus. In this paper, we describe our motivation for organizing the task, our task framework, and the results of participants. Past research has shown that supervised learning is one of the most successful approaches to WSD. However, this approach involves the collection of a large text corpus in which each ambiguous word has been annotated with the correct sense to serve as training data. Due to the expensive annotation pro- cess, only a handful of manually sense-tagged cor- pora are available. An effort to alleviate the training data bottle- neck is the Open Mind Word Expert (OMWE) project (Chklovski and Mihalcea, 2002) to collect sense-tagged data from Internet users. Data gath- ered through the OMWE project were used in the SENSEVAL-3 English lexical sample task. In that task, WordNet-1.7.1 was used as the sense inven- tory for nouns and adjectives, while Wordsmyth1 was used as the sense inventory for verbs. Another source of potential training data is par- allel texts. Our past research in (Ng et al., 2003; Chan and Ng, 2005) has shown that examples gath- ered from parallel texts are useful for WSD. Briefly, after manually assigning appropriate Chinese trans- lations to each sense of an English word, the English side of a word-aligned parallel text can then serve as the training data, as they are considered to have been disambiguated and “sense-tagged” by the appropri- ate Chinese translations. Using the above approach, we gathered the train- ing and test examples for our task from parallel texts. Note that our examples are collected without manu- ally annotating each individual ambiguous word oc- currence, allowing us to gather our examples in a much shorter time. This contrasts with the setting of the English lexical sample task in previous SENSE- VAL evaluations. In the English lexical sample task of SENSEVAL-2, the sense tagged data were cre- ated through manual annotation by trained lexicog- raphers. In SENSEVAL-3, the data …

Content truncated to 500 words