Identification of Basic Phrases for Kazakh Language using Maximum Entropy Model

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1007–1014, Dublin, Ireland, August 23-29 2014. Identification of Basic Phrases for Kazakh Language using Maximum Entropy Model Gulila Altenbek*,+ Xiaolong Wang* Gulizhada Haisha+ *School of Computer Science and Technology, Harbin Instituteof Technology,150001,China. +College of Information Science and Engineering, Xinjiang University,830046, China. +The Base of Kazakh and Kirghiz Language of National Language Resource Monitoring and Research Centre Minority Languages, Xinjiang, 830046, China. gla@insun.hit.edu.cn, gla@xju.edu.cn, wangxl@insun.hit.edu.cn Abstract This paper proposes the definition, classification and structure of the Kazakh basic phrases, and sets up a framework for classifying them according to their syntactic functions. Meanwhile, the structure of the Kazakh basic phrases were analyzed; and the determination of the Kazakh basic phrases collocation and extraction of the Kazakh basic phrases based on rules were followed. The Maximum Entropy (ME) model uses for the identification of the phrases from texts and achieved a result of automatic identifica- tion of Kazakh phrases with an accuracy of 78.22% based on rules System and additional artificial mod- ification. Design feature of this ME model join rely on templates of Kazakh Word, part of speech, affix- es. Experimental results show that the accuracy rate reached 87.89％． 1 Introduction Automatic phrase identification is an important task in natural language processing. A phrase is a group of words that work together. Phrase recognition is a grammatical unit agent between words and sentences in natural language processing. Phrase identification Parser has been developed for different languages, for example, the Church's Base NP Recognition for English (Church, 1988). The rule-based Model and Maximum Entropy Model (ME) are the most commonly used technology for phrase repre- sentation and parsing. Kazakh Language belongs to the Turkish Language group in the Altaic language family. It is an agglu- tinative language with word structures formed by adding derivational or inflectional affixes to root words. Phrase identification is also an indispensable part for Kazakh information processing. In the past a few year, we have put forward methods for Kazakh morphological analysis, which includes stem extraction, part of speech(POS) tagging, spelling check, etc. Recently, we are working on syntax parsing, analysis of phrase structure, automatic identification of phrase and in-depth analysis of sen- tence structure. Kazakh phrases are syntactic units consisting of two or more than two words. The phrases can be clas- sified into two categories, which are free phrase and fixed phrase. We are exploring methods which are more suitable for shallow syntactic parsing of Kazakh according to the nature of Kazakh language. The research includes a systematic study on information regularity and disambiguation of the Kazakh phrase, and automatic recognition of basic phrases of Kazakh language. We have developed a rule- based method for the automatic recognition of Kazakh basic phrases, and automatic identification of verb phrase, noun phrase and adjective phrase based on maximum entropy in Kazakh language at the same time. Moreover, the ambiguity of structures is also resolved based on rules. This study solves the problem of Kazakh …

Content truncated to 500 words