Disambiguating Main POS tags for Turkish

[ Disambiguating Main POS tags for Turkish Razieh Ehsani Istanbul Technical University Faculty of Computer and Informatics Computer Engineering rehsani@itu.edu.tr Muzaffer Ege Alper National University of Singapore Faculty of Science Department of Statistics and Applied Probability m.ege85@nus.edu.sg G¨uls¸en Eryi˘git Istanbul Technical University Faculty of Computer and Informatics Computer Engineering gulsenc@itu.edu.tr Es¸ref Adalı Istanbul Technical University Faculty of Computer and Informatics Computer Engineering adali@itu.edu.tr ] Abstract This paper presents the results of main part-of-speech tagging of Turkish sentences using Conditional Ran- dom Fields (CRFs). Although CRFs are applied to many different languages for part-of-speech (POS) tagging, Turkish poses interesting challenges to be modeled with them. The challenges include issues related to the statistical model of the problem as well as issues related to computational complexity and scaling. In this paper, we propose a novel model for main-POS tagging in Turkish. Furthermore, we pro- pose some approaches to reduce the computational complexity and allow better scaling characteristics or improve the performance without increased complexity. These approaches are discussed with respect to their advantages and disadvantages. We show that the best approach is competitive with the current state of the art in accuracy and also in training and test durations. The good results obtained imply a good first step towards full morphological disambiguation. 1 Introduction The morphological disambiguation problem for morphologically rich languages differs significantly from the well known POS tagging problem. It is rather an automatic selection process from multiple legal analysis of a given word than the assignment of a POS tag from a predetermined tag set. The possible morphological analyses of a word (generally produced by a morphological analyzer) in such languages are very complex when compared to morphologically simple ones: They consist of the lemma, the main POS tags and the tags related to the inflectional and derivational affixes. The number of the set of possible morphological analyses may sometimes be infinite for some languages such as Turkish. In this study, we focus on the determination of the main POS tags (which will be referred as “POS tagging” from now on) rather than the full disambiguation task. There are few methods for Turkish which directly tackle POS tagging problem. Instead many methods perform a full morphological disambiguation and the POS tags are obtained from the correct parses. In this work, we take a different approach and propose a model which directly tackles the POS tagging problem. While also being useful in its own right, this method is also a first step towards full morphological disambiguation through weighted opinion pooling approach [16]. To give a sense of the problem at hand and the general morphological disambiguation, we have measured the ambiguity corresponding to the POS tagging and Morphological Disambiguation problems. About 27% of the words in our corpus are ambiguous in terms of its POS tag and random guessing has an expected accuracy of 85%, on the other hand the ambiguity in terms of morphological disambiguation is about %50. The proposed approach in this paper improves the accuracy of POS …

Content truncated to 500 words