A Korean Homonym Disambiguation System Based on Statistical Model Using Weights

A Korean Homonym Disambiguation System Based on Statistical Model Using weights Jun-Su Kim Wang-Woo Lee Chang-Hwan Kim Cheol-young Ock Dept. of Computer Engineering & Information Technology, University of Ulsan 680-749, San29, Mugeo-dong, Nam-gu Ulsan, Korea {j skim, wwlee, edel}@cic.ulsan.ac.kr okcy@uou.ulsan.ac.kr Abstract A homonym could be disambiguated by another words in the context as nouns, predicates used with the homonym. This paper using semantic information (co-occurrence data) obtained from definitions of part of speech (POS) tagged UMRD-S 1). In this research, we have analyzed the result of an experiment on a homonym disambiguation system based on statistical model, to which Bayes' theorem is applied, and suggested a model established of the weight of sense rate and the weight of distance to the adjacent words to improve the accuracy. The result of applying the homonym disambiguation system using semantic information to disambiguating homonyms appearing on the dictionary definition sentences showed average accuracy of 98.32% with regard to the most frequent 200 homonyms. We selected 49 (31 substantives and 18 predicates) out of the 200 homonyms that were used in the experiment, and performed an experiment on 50,703 sentences extracted from Sejong Project tagged corpus (i.e. a corpus of morphologically analyzed words) of 3.5 million words that includes one of the 49 homonyms. The result of experimenting by assigning the weight of sense rate(prior probability) and the weight of distance concerning the 5 words at the front/behind the homonym to be disambiguated showed better accuracy than disambiguation systems based on existing statistical models by 2.93%. 1� Introduction The ambiguity, which is the most difficult problem in natural language processing (NLP), occurs inevitably in every analysis process including morphological analysis and syntactic analysis. Ambiguity problems occurring at some parts have been resolved to some degree. As the studies on semantic and discourse analysis are becoming active, more effort is being made to research schemes for word sense disambiguation (WSD). WSD refers to disambiguating the sense of a word contextually suitable for the sentence when it is used with two or more different meanings in sentences. [1, 3, 4, 5, 8] Studies for solving ambiguity are largely grouped into methods using dictionaries according to the pattern of learning data and ones using a corpus. In terms of methodology, there are largely methods using rules, ones using probability statistics, and ones using semantic hierarchy structure. A method using dictionaries is disadvantageous in that it is difficult to reflect the dynamic characteristic of a language [10, 11], while advantageous in that it can abstract the detailed information of word senses [2, 3]. To disambiguate using a corpus, a large-sized semantic tagged corpus is required. However, a high-quality corpus is hard to find, and costly and time-consuming to build. But the method reflects the dynamic characteristic of a language. In the thesis, we abstract semantic information from a dictionary definition corpus based on the method suggested in J. Huh (2000)[3] research. We will study a plan to utilize the semantic information in the homonym disambiguation model based on …

Content truncated to 500 words