An Empirical Study of the Domain Dependence of Supervised Word Disambiguation Systems

An Empirical Study of the Domain Dependence of Supervised Word Sense Disambiguation Systems* Gerard Escudero, Lluis M~trquez~ and German Rigau TALP Research Center. LSI Department. Universitat Polit~cnica de Catalunya (UPC) Jordi Girona Salgado 1-3. E-08034 Barcelona. Catalonia {escudero, lluism, g. rigau}@isi, upc. es Abstract This paper describes a set of experiments car- ried out to explore the domain dependence of alternative supervised Word Sense Disam- biguation algorithms. The aim of the work is threefold: studying the performance of these algorithms when tested on a different cor- pus from that they were trained on; explor- ing their ability to tune to new domains, and demonstrating empirically that the Lazy- Boosting algorithm outperforms state-of-the- art supervised WSD algorithms in both previ- ous situations. Keywords: Cross-corpus evaluation of Ni_P systems, Word Sense Disambiguation, Super- vised Machine Learning 1 Introduction Word Sense Disambiguation (WSD) is the problem of assigning the appropriate meaning (sense) to a given word in a text or discourse. Resolving the ambiguity of words is a central problem for large scale language understand- ing applications and their associate tasks (Ide and V4ronis, 1998), e.g., machine transla- tion, information retrieval, reference resolu- tion, parsing, etc. WSD is one of the most important open problems in NLP. Despite the wide range of approaches investigated and the large effort devoted to tackle this problem, to date, no large-scale broad-coverage and highly accu- rate WSD system has been built --see the main conclusions of the first edition of Sen- sEval (Kilgarriff and Rosenzweig, 2000). One of the most successful current lines of research is the corpus-based approach in " This research has been partially funded by the Span- ish Research Department (CICYT's project TIC98- 0423-C06). by the EU Commission (NAMIC IST- 1999-12392), and by the Catalan Research Depart- ment (CIRIT's consolidated research group 1999SGR- 150 and CIRIT's grant 1999FI 00773). which statistical or Machine Learning (ML) al- gorithms are applied to learn statistical mod- els or classifiers from corpora in order to per- form WSD. Generally, supervised approaches 1 have obtained better results than unsuper- vised methods on small sets of selected am- biguous words, or artificial pseudo-words. Many standard M L algorithms for supervised learning have been applied, such as: Decision Lists (¥arowsky, 1994; Agirre and Martinez, 2000), Neural Networks (Towell and Voorhees, 1998), Bayesian learning (Bruce and Wiebe, 1999), Exemplar-Based learning (Ng, 1997a; Fujii et al., 1998), Boosting (Escudero et al., 2000a), etc. Unfortunately, there have been very few direct comparisons between alterna- tive methods for WSD. In general, supervised learning presumes that the training examples are somehow re- flective of the task that will be performed by the trainee on other data. Consequently, the performance of such systems is commonly es- timated by testing the algorithm on a separate part of the set of training examples (say 10- 20% of them), or by N-fold cross-validation, in which the set of examples is partitioned into N disjoint sets (or folds), and the training- test procedure is repeated N times using all combinations of …

Content truncated to 500 words