Regularized Least-Squares classification for Word Sense Disambiguation

Regularized Least-Squares Classification for Word Sense Disambiguation Marius Popescu Department of Computer Science, University of Bucharest Str. Academiei 14 70109 Bucharest, Romania, mpopescu@phobos.cs.unibuc.ro Abstract The paper describes RLSC-LIN and RLSC- COMB systems which participated in the Senseval-3 English lexical sample task. These systems are based on Regularized Least-Squares Classification (RLSC) learning method. We describe the reasons of choosing this method, how we applied it to word sense disambigua- tion, what results we obtained on Senseval- 1, Senseval-2 and Senseval-3 data and discuss some possible improvements. 1 Introduction Word sense disambiguation can be viewed as a classification problem and one way to ob- tain a classifier is by machine learning methods. Unfortunately, there is no single one universal good learning procedure. The No Free Lunch Theorem assures us that we can not design a good learning algorithm without any assump- tions about the structure of the problem. So, we start by trying to find out what are the par- ticular characteristics of the learning problem posed by the word sense disambiguation. In our opinion, one of the most important particularities of the word sense disambiguation learning problem, seems to be the dimensional- ity problem, more specifically the fact that the number of features is much greater than the number of training examples. This is clearly true about data in Senseval-1, Senseval-2 and Senseval-3. One can argue that this happens because of the small number of training exam- ples in these data sets, but we think that this is an intrinsic propriety of learning task in the case of word sense disambiguation. In word sense disambiguation one important knowledge source is the words that co-occur (in local or broad context) with the word that had to be disambiguated, and every different word that appears in the training examples will be- come a feature. Increasing the number of train- ing examples will increase also the number of different words that appear in the training ex- amples, and so will increase the number of fea- tures. Obviously, the rate of growth will not be the same, but we consider that for any reason- able number of training examples (reasonable as the possibility of obtaining these training exam- ples and as the capacity of processing, learning from these examples) the dimension of the fea- ture space will be greater. Actually, the high dimensionality of the fea- ture space with respect to the number of exam- ples is a general scenario of learning in the case of Natural Language Processing tasks and word sense disambiguation is one of these examples. In such situations, when the dimension of the feature space is greater than the number of training examples, the potential for over- fitting is huge and some form of regulariza- tion is needed. This is the reason why we chose to use Regularized Least-Squares Classifi- cation (RLSC) (Rifkin, 2002; Poggio and Smale, 2003), a method of learning based on kernels and Tikhonov regularization. In the next section we explain what source of information we used and …

Content truncated to 500 words