Large Scale Collocation Data and Their Application to Japanese Word Processor Technology

Large Scale Collocation Data and Their Application to Japanese Word Processor Technology Yasuo Koymna, Masako Yasutake, Kenji Yoshimura and Kosho Shudo Institute for Informalion and Conlrol Systmas, Fukuoka University N ~ Fukuoka, 814-0180 Japan koymm@aisott co.jp, yasutake@helio.tt fukuoka-u.ac.jp, yosimura@flsmtl.fukuoka ac.jp, shudo@flstm.tt fukuoka-u.ac.jp abstract Word processors or computers used in Japan employ Japanese input method through key- board stroke combined with Kana (phonetic) character to Kanji (ideographic, Chinese) char- acter conversion technology. The key factor of Kana-to-Kanji conversion technology is how to raise the accuracy of the conversion through the homophone processing, since we have so many homophonic Kanjis. In this paper, we report the results of our Kana-to-Kanji conver- sion experiments which embody the homo- phone processing based on large scale colloca- tion data. It is shown that approximately 135,000 collocations yield 9.1% raise of the conversion accuracy compared with the pro- totype system which has no collocation data. 1. Introduction Word processors or computers used in Japan ordi- narily employ Japanese input method through key- board stroke combined ~ with Kana (phonetic) to Kanji (ideographic, Chinese) character conversion technology. The Kana-to-Kanji conversion is per- formed by the morphological analysis on the input Kana siring with no space between words. Word- or phrase-segmentation is carried out by the analysis to identify the substring of the input which has to be converted from Kana to Kanji. Kana-Kanji mixed string, which is the ordinary form of Japanese writ- ten text, is obtained as the final result. The major issue of this technology lies in raising the accuracy of the segmentation and the homophone processing to select the correct Kanji among many homophonic candidates. The conventional methodology for processing ho- mophones have used the function that gives the pri- ority to the word which was used lastly or to the high frequency word. In fact, however, this method sometimes tends to cause inadequate conversion due to the lack of consideration of the semantic consis- tency of the word concurrence. While it is difficult to employ the syntactic or semantic processing in earnest for the word processor from the cost vs. performance viewpoints, for example, the following trials to improve the conversion accuracy have been reported: Employing the case-frame to check the semantic consistency of combination of words [Oshima, Y. et al., 1986]. Employing the neural net- work to describe the consistency of the concurrence of words [Kobayashi, T. et al.,1992], Making a con- currence dictionary for the specific topic or field, and giving the priority to the word which is in the dictionary when the topic is identified [Yamamoto, K. et al., 1992]. In any of these studies, however, many problems are left unsolved in realizing its practical system. Besides these semantic or quasi-semantic gadgets, we think it much more practical and effective to use surface level resources, namely, to use extensively the collocation. But how many collocations contrib- ute to the accuracy of Kana-to-Kanji conversion is not known yet. In this paper, we present some results of our ex- periments of Kana-to-Kanji …

Content truncated to 500 words