Applying Alternating Structure Optimization to Word Sense Disambiguation

Proceedings of the 10th Conference on Computational Natural Language Learning (CoNLL-X), pages 77–84, New York City, June 2006. c⃝2006 Association for Computational Linguistics Applying Alternating Structure Optimization to Word Sense Disambiguation Rie Kubota Ando IBM T.J. Watson Research Center Yorktown Heights, NY 10598, U.S.A. rie1@us.ibm.com Abstract This paper presents a new application of the recently proposed machine learning method Alternating Structure Optimiza- tion (ASO), to word sense disambiguation (WSD). Given a set of WSD problems and their respective labeled examples, we seek to improve overall performance on that set by using all the labeled exam- ples (irrespective of target words) for the entire set in learning a disambiguator for each individual problem. Thus, in effect, on each individual problem (e.g., disam- biguation of “art”) we benefit from train- ing examples for other problems (e.g., disambiguation of “bar”, “canal”, and so forth). We empirically study the effective use of ASO for this purpose in the multi- task and semi-supervised learning config- urations. Our performance results rival or exceed those of the previous best sys- tems on several Senseval lexical sample task data sets. 1 Introduction Word sense disambiguation (WSD) is the task of assigning pre-defined senses to words occurring in some context. An example is to disambiguate an oc- currence of “bank” between the “money bank” sense and the “river bank” sense. Previous studies e.g., (Lee and Ng, 2002; Florian and Yarowsky, 2002), have applied supervised learning techniques to WSD with success. A practical issue that arises in supervised WSD is the paucity of labeled examples (sense-annotated data) available for training. For example, the train- ing set of the Senseval-21 English lexical sample 1http://www.cs.unt.edu/ ~rada/senseval/. WSD systems have task has only 10 labeled training examples per sense on average, which is in contrast to nearly 6K training examples per name class (on average) used for the CoNLL-2003 named entity chunking shared task2. One problem is that there are so many words and so many senses that it is hard to make available a suf- ficient number of labeled training examples for each of a large number of target words. On the other hand, this indicates that the total number of available labeled examples (irrespective of target words) can be relatively large. A natural question to ask is whether we can effectively use all the labeled examples (irrespective of target words) for learning on each individual WSD problem. Based on these observations, we study a new application of Alternating Structure Optimization (ASO) (Ando and Zhang, 2005a; Ando and Zhang, 2005b) to WSD. ASO is a recently proposed ma- chine learning method for learning predictive struc- ture (i.e., information useful for predictions) shared by multiple prediction problems via joint empiri- cal risk minimization. It has been shown that on several tasks, performance can be significantly im- proved by a semi-supervised application of ASO, which obtains useful information from unlabeled data by learning automatically created prediction problems. In addition to such semi-supervised learn- ing, this paper explores ASO multi-task learning, which learns a number …

Content truncated to 500 words