Improving POS Tagging Using Machine-Learning Techniques

Improving POS Tagging Using Machine-Learning Techniques Llufs Mhrquez 1, Horacio Rodrfguez 2, Josep Carmona 1 and Josep Montolio 1 1 TALP Research Center. Dep. LSI - Universitat Polit~cnica de Catalunya c/Jordi Girona 1-3. 08034 Barcelona. Catalonia lluism@l si .upc. es 2 Dep. IMA - Universitat de Girona Horacio. Rodriguez@ima. udg. es Abstract In this paper we show how machine learning techniques for constructing and combining sev- eral classifiers can be applied to improve the accuracy of an existing English POS tagger (MSxquez and Rodrfguez, 1997). Additionally, the problem of data sparseness is also addressed by applying a technique of generating convez pseudo-data (Breiman, 1998). Experimental re- sults and a comparison to other state-of-the- art tuggers are reported. Keywords: POS Tagging, Corpus-based mod- eling, Decision Trees, Ensembles of Classifiers. 1 Introduction The study of general methods to improve the performance in classification tasks, by the com- bination of different individual classifiers, is a currently very active area of research in super- vised learning. In the machine learning (ML) literature this approach is known as ensemble, stacked, or combined classifiers. Given a classi- fication problem, the main goal is to construct several independent classifiers, since it has been proven that when the errors committed by indi- vidual classifiers are uncorrelated to a sufficient degree, and their error rates are low enough, the resulting combined classifier performs bet- ter than all the individual systems (Ali and Paz- zani, 1996; Tumer and Ghosh, 1996; Dietterich, 1997). Several methods have been proposed in order to construct ensembles of classifiers that make uncorrelated errors. Some of them are general, and they can be applied to any learning algo- rithln, while other are specific to particular al- gorithms. From a different perspective, there exist methods for constructing homogeneous en- sembles, in the sense that a unique learning al- gorithm has been used to acquire each individ- ual classifier, and heterogeneous ensembles that combine different types of learning paradigms 1. Impressive results have been obtained by ap- plying these techniques on the so-called unsta- ble learning algorithms (e.g. induction of deci- sion trees, neural networks, rule-induction sys- tems, etc.). Several applications to real tasks have been performed, and, regarding NLP, we find ensembles of classifiers in context-sensitive spelling correction (Golding and Roth, 1999), text categorization (Schapire and Singer, 1998; Blum and Mitchell, 1998), and text filtering (Schapire et al., 1998). Combination of classi- tiers have also been applied to POS tagging. For instance, van Halteren (1996) combined a num- ber of similar tuggers by way of a straightfor- ward majority vote. More recently, two parallel works (van Halteren et al., 1998; Brill and Wu, 1998) combined, with a remarkable success, the output of a set of four tuggers based on different principles and feature modelling. Finally, in the work by MSxquez et al. (1998) the combination of taggers is used in a bootstrapping algorithm to train a part of speech tagger from a limited amount of training material. The aim of the present work is to improve an …

Content truncated to 500 words