SENSEVAL-2 Japanese Dictionary Task

SENSEVAL-2 Japanese Dictionary Task Kiyoaki Shirai School of Information Science, Japan Advanced Institute of Science and Technology kshirai@jaist.ac.jp Abstract This paper reports an overview of the SENSEVAL-2 Japanese dictionary task. It was a lexical sample task, and word senses are de- fined according to a Japanese dictionary, the Iwanami Kokugo Jiten. The Iwanami Kokugo Jiten and a training corpus were distributed to all participants. The number of target words was 100, 50 nouns and 50 verbs. One hundred instances of each target word were provided, making for a total of 10,000 instances for eval- uation. Seven systems of three organizations participated in this task. 1 Introduction In SENSEVAL-2, there are two Japanese tasks, a translation task and a dictionary task. This paper describes the details of the dictionary task. First of all, let me introduce an overview of the Japanese dictionary task. This task is a lexical sample task. Word senses were defined according to the Iwanami Kokugo Jiten (Nishio et aL, 1994), a Japanese dictionary published by Iwanami Shoten. It was distributed to all par- ticipants as a sense inventory. Training data, a corpus consisting of 3,000 newspaper articles and manually annotated with sense IDs, was also distributed to participants. For evaluation, we distributed newspaper articles with marked target words as test documents. Participants were required to assign one or more sense IDs to each target word, optionally with associated probabilities. The number of target words was 100, 50 nouns and 50 verbs. One hundred in- stances of each target word were provided, mak- ing for a total of 10,000 instances. In what follows, Section 2 describes details of data used in the Japanese dictionary task. Section 3 describes the process to construct the 33 gold standard data, including the analysis of inter-tagger agreement. Section 4 briefly intro- duces participating systems and their results. Finally, Section 5 concludes this paper. 2 Data In the Japanese dictionary task, three data were distributed to all participants: sense inventory, training data and evaluation data. 2.1 Sense Inventory As described in Section 1, word senses are de- fined according to a Japanese dictionary, the Iwanami Kokugo Jiten. The number of head- words and word senses in the I wanami Kokugo Jiten is 60,321 and 85,870, respectively. Figure 1 shows an example of word sense de- scriptions in the Iwanami Kokugo Jiten, the sense set of the Japanese noun "MURI." MURI 1. lack of reasonableness 1-a. something not to be rational, not to be sen- sible [kimi ga okoru no wa MURI mo nai (It is natural for you to be angry)] 1-b. to do something compulsorily [ sigoto no MURI de byouki ni naru (I become ill from overwork)] Figure 1: Sense set of "MURI" As shown in Figure 1, there are hierarchical structures in word sense descriptions. For ex- ample, word sense 1 subsumes 1-a and 1-b. The number of layers of hierarchy in the I wanami Kokugo Jiten is at most 3. Word sense dis- tinctions in the lowest level are …

Content truncated to 500 words