Rule-based Approach to Korean Morphological Disambiguation Supported by Statistical Method

Language, Information and Computation(PACLIC 11), 1996, 237-246 Rule-based Approach to Korean Morphological Disambiguation Supported by Statistical Method Min-Jung Kim* Hyuk-Chul Kwon * Ae-Sun Yoon** Dept. of Computer Science*, Dept. of French' Pusan National University, Korea {mjkim, hckwon, asyoon)@bandi.cs.pusan.ac.kr ABSTRACT Korean as an agglutinative language shows its proper types of difficulties in morphological disambiguation, since a large number of its ambiguities comes from the stemming while most of ambiguities in French or English are related to the categorization of a morpheme. The current Korean morphological disambiguation systems adopt mainly statistical methods and some of them use rules in the postprocess. In our approach, the morphological analyzer reduces the number of the candidate morpheme strings using adjacency conditions when it analyses a word into morpheme strings. And then the disambiguation depends on rules and statistics successively. As for the rules, the partial parsing using finite state automata decides the compatibility of each pair of words: a negative value is assigned if a word can not co-occur with another word, while a positive value is given if they are compatible. After applying all the rules related to the word, our system chooses only the positively valued strings. When more than two strings still have same value, the priority in the context is decided by the statistics in the next stage. The accuracy of our approach as Korean tagging system is about 97.1% and it may yeild a better result than the Korean morphological disambiguation systems. I. Introduction Compared to French or English, Korean as an agglutinative language shows its proper types of difficulties in morphological disambiguation, since a large number of its ambiguities comes from the stemming while most of ambiguities in French or English are related to the categorization of a morpheme[1]. In the case that a Korean word can be analyzed into several different morpheme strings[2], it is not easy to decide which one is the most compatible with the context. And what is worse, spacing rules of the Korean orthography are optional in some cases or they are easily violated in the other cases: it depends 237 on authors to give spaces or not between two nouns in noun compounds; more than 10% of words in newspapers violate the spacing rules for the lack of space[3,4]. The current Korean morphological disambiguation systems adopt mainly statistical methods and some of them use rules in the postprocess to filter out incompatible strings[5,6,7]. The statistical approach can, though, give only positive clues of adjacency of two morphemes and it fails to predict precisely their mutual exclusions which play an important role in morphological disambiguation. In our approach, the morphological analyzer reduces already the number of the candidate morpheme strings using adjacency conditions when it analyses a word into morpheme strings. And then the disambiguation depends on rules and statistics successively. As for the rules, the partial parsing using finite state automata decides the compatibility of each pair of morphemes: a negative value is assigned if a morpheme can not co-occur with another morpheme, …

Content truncated to 500 words