A Hybrid Morphological Disambiguation System for Turkish

International Joint Conference on Natural Language Processing, pages 1230–1236, Nagoya, Japan, 14-18 October 2013. A Hybrid Morphological Disambiguation System for Turkish Mucahid Kutlu Dept. of Computer Science & Eng., Ohio State University, Ohio, USA kutlu@cse.ohio-state.edu Ilyas Cicekli Department of Computer Engineering, Hacettepe University, Ankara, Turkey ilyas@cs.hacettepe.edu.tr Abstract In this paper, we propose a morphological dis- ambiguation method for Turkish, which is an agglutinative language. We use a hybrid meth- od, which combines statistical information with handcrafted rules and learned rules. Five different steps are applied for disambiguation. In the first step, the most likely tags of words are selected. In the second step, we use hand- crafted rules to constrain possible parses or se- lect the correct parse. Next, the most likely tags are selected for still ambiguous words ac- cording to the suffixes of the words that are unseen in the training corpus. Then, we use transformation-based rules that are learned by a variation of Brill tagger. If the word is still ambiguous, we use some heuristics for the dis- ambiguation. We constructed a hand-tagged dataset for training and applied a ten-fold cross validation with this dataset. We obtained 93.4% accuracy on the average when whole morphological parses are considered in calcu- lation. The accuracy increased to 94.1% when only part-of-speech tags and inflections of last derivations are considered. Our accuracy is 96.9% in terms of part-of-speech tagging. 1 Introduction Digital text sources increase every day and peo- ple can reach digital sources via Internet. The automatic processing of these sources becomes crucial in order to use and manage them effec- tively. Many NLP researchers work on different topics like summarization of texts, translation between natural languages, information extrac- tion, etc. However, working on natural languages has difficulties due to their ambiguous natures. Since constraining possible morphological parses of words with disambiguation methods reduces the ambiguity problem, morphological disam- biguation is crucial for performing better opera- tions on texts. In this paper, we propose a method for mor- phological disambiguation of Turkish texts. Turkish is an agglutinative language. Agglutina- tive languages generate words by joining affixes together and each affix represents one unit of meaning. This property loads many meanings to a single word. Since suffixes can increase the ambiguity by generating totally different words, the morphological analysis of words with many suffixes is much harder. Our hybrid system uses statistical knowledge, learned transformation-based rules, handcrafted rules and some heuristics in the disambiguation process. In our system, we first obtain the statis- tical information about words and suffixes like frequencies of their corresponding morphological parses and learn transformation-based rules. In disambiguation, we use an iterative approach that applies techniques from the most reliable tech- nique to the less reliable technique. First, if the word exists in our training set, we select the morphological parse with the highest frequency. Then, we consider handcrafted rules of Super- visedTagger software (Daybelge and Cicekli, 2007) for disambiguation. Next, we use statisti- cal information about suffixes and select the morphological parse with the highest …

Content truncated to 500 words