A global model for joint lemmatization and part-of-speech prediction

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 486–494, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP A global model for joint lemmatization and part-of-speech prediction Kristina Toutanova Microsoft Research Redmond, WA 98052 kristout@microsoft.com Colin Cherry Microsoft Research Redmond, WA 98052 colinc@microsoft.com Abstract We present a global joint model for lemmatization and part-of-speech predic- tion. Using only morphological lexicons and unlabeled data, we learn a partially- supervised part-of-speech tagger and a lemmatizer which are combined using fea- tures on a dynamically linked dependency structure of words. We evaluate our model on English, Bulgarian, Czech, and Slovene, and demonstrate substantial im- provements over both a direct transduction approach to lemmatization and a pipelined approach, which predicts part-of-speech tags before lemmatization. 1 Introduction The traditional problem of morphological analysis is, given a word form, to predict the set of all of its possible morphological analyses. A morpho- logical analysis consists of a part-of-speech tag (POS), possibly other morphological features, and a lemma (basic form) corresponding to this tag and features combination (see Table 1 for examples). We address this problem in the setting where we are given a morphological dictionary for training, and can additionally make use of un-annotated text in the language. We present a new machine learn- ing model for this task setting. In addition to the morphological analysis task we are interested in performance on two subtasks: tag-set prediction (predicting the set of possible tags of words) and lemmatization (predicting the set of possible lemmas). The result of these sub- tasks is directly useful for some applications.1 If we are interested in the results of each of these two 1Tag sets are useful, for example, as a basis of sparsity- reducing features for text labeling tasks; lemmatization is useful for information retrieval and machine translation from a morphologically rich to a morphologically poor language, where full analysis may not be important. subtasks in isolation, we might build independent solutions which ignore the other subtask. In this paper, we show that there are strong de- pendencies between the two subtasks and we can improve performance on both by sharing infor- mation between them. We present a joint model for these two subtasks: it is joint not only in that it performs both tasks simultaneously, sharing in- formation, but also in that it reasons about multi- ple words jointly. It uses component tag-set and lemmatization models and combines their predic- tions while incorporating joint features in a log- linear model, defined on a dynamically linked de- pendency structure of words. The model is formalized in Section 5 and eval- uated in Section 6. We report results on English, Bulgarian, Slovene, and Czech and show that joint modeling reduces the lemmatization error by up to 19%, the tag-prediction error by up to 26% and the error on the complete morphological analysis task by up to 22.6%. 2 Task formalization The main task that we would like to solve is as follows: given …

Content truncated to 500 words