Japanese Word Sense Disambiguation using the Simple Bayes and Support Vector Machine Methods

Japanese word sense disambiguation using the simple Bayes and support vector machine methods Masaki Murata, Masao Utiyama, Kiyotaka Uchimoto, Qing Ma, and Hitoshi !sahara Communications Research Laboratory 2-2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, Japan Abstract We submitted four systems to the Japanese dictionary-based lexical-sample task of SENSEVAL-2. They were i) the support vector machine method ii) the simple Bayes method, iii) a method combining the two, and iv) a method combining two kinds of each. The combined methods obtained the best precision among the submitted systems. After the contest, we tuned the parameter used in the simple Bayes method, and it obtained higher preciSIOn. An explanation of these systems used in Japanese word sense disambiguation was provided. 1 Introduction We participated in the Japanese dictionary- based lexical-sample task of the SENSEVAL-2 contest. We used machine learning approaches and submitted four systems. After the con- test, we tuned the parameter used in the simple Bayes method and carried out additional exper- iments. In this paper, we explain the systems and their experimental results. 2 Task Descriptions The test data included 10,000 instances for eval- uation. The RWC corpus (Shirai et al., 2001) was given as the training data. It was made from 3000 articles published in the Mainichi Newspaper. The nouns, verbs, and adjectives (the total number of which was about 150,000) were assigned sense tags defined on the basis of the Iwanami dictionary. The purpose of this task was to estimate the sense of a word by us- ing its context. 3 Methods Because the word sense assigned to each word is dependent on the word itself, estimations 135 were conducted using machine learning meth- ods for each word. That is, we constructed as many learning machines as there were individ- ual words. We used the simple Bayes and support vec- tor machine methods as the machine learning method. 1 In this section, we explain each of the machine learning methods and then explain the method combining several of them. 3.1 Simple Bayes Method This method estimates probability based on the Bayes theory. The category (i.e., the sense tag) with the highest probability is judged to be the desired one. This is a basic approach to the disambiguation of word sense. The probability of category a appearing in context b is defined as: p(alb) p(a) p(b) p(bla) (1) p(a) IT- p(b) . p(f;ia), (2) ' where context b is a set of features fj ( E F, 1 ~ j ~ k) that is defined in advance. p(b) is the probability of context b, which is not calculated because it is a constant and is not dependent on category a. p(a) and jj(fila) are the prob- abilities estimated by using the training data and indicate the probability of the occurrence of category a in the examples of the training data and the probability of feature fi occur- ring, given category a, respectively. When we use the maximum likelihood estimation to cal- culate p(fiia), which often has a value of 0 …

Content truncated to 500 words