Tagging and Morphological Disambiguation of Turkish Text

Tagging and Morphological Disambiguation of Turkish Text Kemal Oflazer and ilker Kuruiiz Department of Computer Engineering and Information Science Bilkent University Bilkent, Ankara, TURKEY {ko,kuruoz}~cs.bilkent.edu.tr Abstract Automatic text tagging is an important component in higher level analysis of text corpora, and its output can be used in many natural language processing applica- tions. In languages like Turkish or Finnish, with agglutinative morphology, morpholog- ical disambiguation is a very crucial pro- cess in tagging, as the structures of many lexical forms are morphologically ambigu- ous. This paper describes a POS tagger for Turkish text based on a full-scale two-level specification of Turkish morphology that is based on a lexicon of about 24,000 root words. This is augmented with a multi- word and idiomatic construct recognizer, and most importantly morphological dis- ambiguator based on local neighborhood constraints, heuristics and limited amount of statistical information. The tagger also has functionality for statistics compilation and fine tuning of the morphological an- alyzer, such as logging erroneous morpho- logical parses, commonly used roots, etc. Preliminary results indicate that the tag- ger can tag about 98-99% of the texts ac- curately with very minimal user interven- tion. Furthermore for sentences morpho- logically disambiguated with the tagger, an LFG parser developed for Turkish, gener- ates, on the average, 50% less ambiguous parses and parses almost 2.5 times faster. The tagging functionality is not specific to Turkish, and can be applied to any lan- guage with a proper morphological analysis interface. 1 Introduction As a part of large scale project on natural language processing for Turkish, we have undertaken the de- velopment of a number of tools for analyzing Turk- ish text. This paper describes one such tool - a text tagger for Turkish. The tagger is based on a full scale two-level morphological specification of Turk- ish (Oflazer, 1993), implemented on the PC-KIMMO environment (Antworth, 1990). In this paper, we de- scribe the functionality and the performance of our tagger along with various techniques that we have employed to deal with various sources of ambigui- ties. 2 Tagging Text Automatic text tagging is an important step in dis- covering the linguistic structure of large text cor- pora. Basic tagging involves annotating the words in a given text with various pieces of information, such as part-of-speech and other lexical features. Part-of-speech tagging facilitates higher-level analy- sis, such as parsing, essentially by performing a cer- tain amount of ambiguity resolution using relatively cheaper methods. The most important functionality of a tagger is the resolution of the structure and parts-of-speech of the lexical items in the text. This, however, is not a very trivial task since many words are in general am- biguous in their part-of-speech for various reasons. In English, for example a word such as make can be verb or a noun. In Turkish, even though there are ambiguities of such sort, the agglutinative na- ture of the language usually helps resolution of such ambiguities due to morphotactical restrictions. On the other hand, this very nature introduces another …

Content truncated to 500 words