Dependency of context-based Word Sense Disambiguation from representation and domain complexity

Dependency of context-based Word Sense Disambiguation from representation and domain complexity Paola Velardi Dipartimento di Scienze dell' Informazione University "La Sapienza" Roma Velardi@dsi.uniromal.it Alessandro Cucchiarelli Istituto di Informatica University of Ancona Ancona alex@inform.unian.it Abstract Word Sense Disambiguation (WSD) is a central task in the area of Natural Language Processing. In the past few years several context-based probabilistic and machine learning methods for WSD have been presented in literature. However, an important area of research that has not been given the attention it deserves is a formal analysis of the parameters affecting the performance of the learning task faced by these systems. Usually performance is estimated by measuring precision and recall of a specific algorithm for specific test sets and environmental conditions. Therefore, a comparison among different learning systems and an objective estimation of the difficulty of the learning task is extremely difficult. In this paper we propose, in the framework of Computational Learning theory, a formal analysis of the relations between accuracy of a context-based WSD system, the complexity of the context representation scheme, and the environmental conditions (e.g. the complexity of language domain and concept inventory ) . 1 Introduction In the literature (see Computational Linguistics (1998) for some recent results), there is a rather vast repertoire of supervised and unsupervised learning algorithms for WSD, most of which are based on a formal characterization of the surrounding context of a word or linguistic concept 1, and a function f to compute the membership of a word to a category, given its context in running texts. Despite the rich literature, none of these algorithms exhibit an "acceptable" performance with reference to the needs of real-world computational task (e.g. Information Retrieval, Information Extraction, Machine Translation etc.), except for particularly straightforward cases. A very interesting WSD experiment is Senseval (1998), a large:-scale exercise in evaluating WSD programs. One of the objectives of this experiment was to identify correlations between performance of the various systems and the parameters of the WSD task. Though the scoring of systems appears sensitive to certain factors, such as the degree of polysemy and the entropy of sense distributions, these correlations could not be consistently observed. There are words with fewer senses (e.g. bet, consume, generous) causing troubles to most systems, while there are words with a very high polysemy and entropy (e.g. shake) on which all systems obtain good performance. The justification that the Senseval coordinator Adam Kilgariff provides for shake is very interesting in the light of what we will discuss later in this paper: "The items (means contexts) for shake involve multi-word expressions, such as shake one's head. (...) Over 50% of the items for shake involve some multi-word expression or other." In other words, the contexts for shake are very 1 The inventory of linguistic concepts is usually extracted from on-line resources like WordNet, the Longman dictionary (LDOCE), or HECTOR. 28 repetitive in the training set, therefore all systems could easily learn a sense discrimination model. Furthermore, in Senseval (but also in other reported …

Content truncated to 500 words