SemEval-2010 Task: Japanese WSD

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 69–74, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics SemEval-2010 Task: Japanese WSD Manabu Okumura Tokyo Institute of Technology oku@pi.titech.ac.jp Kiyoaki Shirai Japan Advanced Institute of Science and Technology kshirai@jaist.ac.jp Kanako Komiya Tokyo University of Agriculture and Technology kkomiya@cc.tuat.ac.jp Hikaru Yokono Tokyo Institute of Technology yokono@lr.pi.titech.ac.jp Abstract An overview of the SemEval-2 Japanese WSD task is presented. It is a lexical sample task, and word senses are defined according to a Japanese dictionary, the Iwanami Kokugo Jiten. This dictionary and a training corpus were distributed to participants. The number of target words was 50, with 22 nouns, 23 verbs, and 5 adjectives. Fifty instances of each target word were provided, consisting of a to- tal of 2,500 instances for the evaluation. Nine systems from four organizations par- ticipated in the task. 1 Introduction This paper reports an overview of the SemEval- 2 Japanese Word Sense Disambiguation (WSD) task. It can be considered an extension of the SENSEVAL-2 Japanese monolingual dictionary- based task (Shirai, 2001), so it is a lexical sam- ple task. Word senses are defined according to the Iwanami Kokugo Jiten (Nishio et al., 1994), a Japanese dictionary published by Iwanami Shoten. It was distributed to participants as a sense inven- tory. Our task has the following two new charac- teristics: 1. All previous Japanese sense-tagged corpora were from newspaper articles, while sense- tagged corpora were constructed in English on balanced corpora, such as Brown corpus and BNC corpus. The first balanced corpus of contemporary written Japanese (BCCWJ corpus) is now being constructed as part of a national project in Japan (Maekawa, 2008), and we are now constructing a sense-tagged corpus based on it. Therefore, the task will use the first balanced Japanese sense-tagged corpus. Because a balanced corpus consists of docu- ments from multiple genres, the corpus can be divided into multiple sub-corpora of a genre. In supervised learning approaches on word sense disambiguation, because word sense distribution might vary across different sub-corpora, we need to take into account the genres of training and test corpora. There- fore, word sense disambiguation on a bal- anced corpus requires tackling a kind of do- main (genre) adaptation problem (Chang and Ng, 2006; Agirre and de Lacalle, 2008). 2. In previous WSD tasks, systems have been required to select a sense from a given set of senses in a dictionary for a word in one con- text (an instance). However, the set of senses in the dictionary is not always complete. New word senses sometimes appear after the dic- tionary has been compiled. Therefore, some instances might have a sense that cannot be found in the dictionary’s set. The task will take into account not only the instances that have a sense in the given set but also the in- stances that have a sense that cannot be found in the set. In the latter case, systems should output that the instances have a sense that is …

Content truncated to 500 words