Unsupervised learning of word sense disambiguation rules by estimating an optimum iteration number in the EM algorithm

Unsupervised learning of word sense disambiguation rules by estimating an optimum iteration number in the EM algorithm Hiroyuki Shinnou Department of Systems Engineering, Ibaraki University 4-12-1 Nakanarusawa, Hitachi, Ibaraki 316-8511 JAPAN shinnou@dse.ibaraki.ac.jp Minoru Sasaki Department of Computer and Information Sciences, Ibaraki University 4-12-1 Nakanarusawa, Hitachi, Ibaraki 316-8511 JAPAN sasaki@cis.ibaraki.ac.jp Abstract In this paper, we improve an unsuper- vised learning method using the Expectation- Maximization (EM) algorithm proposed by Nigam et al. for text classification problems in order to apply it to word sense disambigua- tion (WSD) problems. The improved method stops the EM algorithm at the optimum itera- tion number. To estimate that number, we pro- pose two methods. In experiments, we solved 50 noun WSD problems in the Japanese Dic- tionary Task in SENSEVAL2. The score of our method is a match for the best public score of this task. Furthermore, our methods were con- firmed to be effective also for verb WSD prob- lems. 1 Introduction In this paper, we improve an unsupervised learning method using the Expectation-Maximization (EM) algo- rithm proposed by (Nigam et al., 2000) for text classifi- cation problems in order to apply it to word sense disam- biguation (WSD) problems. The original method works well, but often causes worse classification for WSD. To avoid this, we propose two methods to estimate the opti- mum iteration number in the EM algorithm. Many problems in natural language processing can be converted into classification problems, and be solved by an inductive learning method. This strategy has been very successful, but it has a serious problem in that an in- ductive learning method requires labeled data, which is expensive because it must be made manually. To over- come this problem, unsupervised learning methods using huge unlabeled data to boost the performance of rules learned by small labeled data have been proposed re- cently(Blum and Mitchell, 1998)(Yarowsky, 1995)(Park et al., 2000)(Li and Li, 2002). Among these methods, the method using the EM algorithm proposed by the pa- per(Nigam et al., 2000), which is referred to as the EM method in this paper, is the state of the art. However, the target of the EM method is text classification. It is hoped that this method can be applied to WSD, because WSD is the most important problem in natural language process- ing. The EM method works well in text classification, but often causes worse classification in WSD. The EM method is expected to improve the accuracy of learned rules step by step in proportion to the iteration number in the EM algorithm. However, this rarely happens in prac- tice, and in many cases, the accuracy falls after a certain iteration number in the EM algorithm. In the worst case, the accuracy of the rule learned through only labeled data is degraded by using unlabeled data. To overcome this problem, we estimate an optimum iteration number in the EM algorithm, and in actual learning, we stop the itera- tion of the EM algorithm at the estimated number. If the estimated number …

Content truncated to 500 words