Improving Data Driven Wordclass Tagging by System Combination

Improving Data Driven Wordclass Tagging by System Combination Hans van Halteren Dept. of Language and Speech University of Nijmegen P.O. Box 9103 6500 HD Nijmegen The Netherlands hvh@let.kun.nl Jakub Zavrel, Walter Daelemans Dept. of Computational Linguistics Tilburg University P.O. Box 90153 5000 LE Tilburg The Netherlands Jakub.Zavrel@kub.nl,Walter.Daelemans@kub.nl Abstract In this paper we examine how the differences in modelling between different data driven systems performing the same NLP task can be exploited to yield a higher accuracy than the best indi- vidual system. We do this by means of an ex- periment involving the task of morpho-syntactic wordclass tagging. Four well-known tagger gen- erators (Hidden Markov Model, Memory-Based, Transformation Rules and Maximum Entropy) are trained on the same corpus data. Af- ter comparison, their outputs are combined us- ing several voting strategies and second stage classifiers. All combination taggers outperform their best component, with the best combina- tion showing a 19.1% lower error rate than the best individual tagger. Introduction In all Natural Language Processing (NLP) systems, we find one or more language models which are used to predict, classify and/or interpret language related observa- tions. Traditionally, these models were catego- rized as either rule-based/symbolic or corpus- based/probabilistic. Recent work (e.g. Brill 1992) has demonstrated clearly that this cat- egorization is in fact a mix-up of two distinct Categorization systems: on the one hand there is the representation used for the language model (rules, Markov model, neural net, case base, etc.) and on the other hand the manner in which the model is constructed (hand crafted vs. data driven). Data driven methods appear to be the more popular. This can be explained by the fact that, in general, hand crafting an explicit model is rather difficult, especially since what is being modelled, natural language, is not (yet) well- understood. When a data driven method is used, a model is automatically learned from the implicit structure of an annotated training cor- pus. This is much easier and can quickly lead to a model which produces results with a 'rea- sonably' good quality. Obviously, 'reasonably good quality' is not the ultimate goal. Unfortunately, the quality that can be reached for a given task is limited, and not merely by the potential of the learn- ing method used. Other limiting factors are the power of the hard- and software used to imple- ment the learning method and the availability of training material. Because of these limitations, we find that for most tasks we are (at any point in time) faced with a ceiling to the quality that can be reached with any (then) available ma- chine learning system. However, the fact that any given system cannot go beyond this ceiling does not mean that machine learning as a whole is similarly limited. A potential loophole is that each type of learning method brings its own 'in- ductive bias' to the task and therefore different methods will tend to produce different errors. In this paper, we are concerned with the ques- tion whether these differences …

Content truncated to 500 words