Automatic Labeling of Voiced Consonants for Morphological Analysis of Modern Japanese Literature

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 292–300, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Automatic Labeling of Voiced Consonants for Morphological Analysis of Modern Japanese Literature Teruaki Oka† teruaki-o@is.naist.jp Toshinobu Ogiso‡ togiso@ninjal.jp †Nara Institute of Science and Technology ‡National Institute for Japanese Language and Linguistics Mamoru Komachi† komachi@is.naist.jp Yuji Matsumoto† matsu@is.naist.jp Abstract Since the present-day Japanese use of voiced consonant mark had established in the Meiji Era, modern Japanese lit- erary text written in the Meiji Era of- ten lacks compulsory voiced consonant marks. This deteriorates the performance of morphological analyzers using ordi- nary dictionary. In this paper, we pro- pose an approach for automatic labeling of voiced consonant marks for modern liter- ary Japanese. We formulate the task into a binary classification problem. Our point- wise prediction method uses as its feature set only surface information about the sur- rounding character strings. As a conse- quence, training corpus is easy to obtain and maintain because we can exploit a par- tially annotated corpus for learning. We compared our proposed method as a pre- processing step for morphological analy- sis with a dictionary-based approach, and confirmed that pointwise prediction out- performs dictionary-based approach by a large margin. 1 Introduction Recently, corpus-based approaches have been suc- cessfully adopted in the field of Japanese Linguis- tics. However, the central part of the fields has been occupied by historical research that uses an- cient material, on which fundamental annotations are often not yet available. Despite the limited annotated corpora, re- searchers have developed several morphological analysis dictionaries for past-day Japanese. Na- tional Institute for Japanese Language and Lin- guistics creates Kindai-bungo UniDic,1 a morpho- logical analysis dictionary for modern Japanese 1http://www2.ninjal.ac.jp/lrc/index.php?UniDic literary text,2 which achieves high performance on analysis for existing electronic text (e.g. Aozora- bunko, an online digital library of freely available books and work mainly from out-of-copyright ma- terials). However, the performance of morphological an- alyzers using the dictionary deteriorates if the text is not normalized, because these dictionaries often lack orthographic variations such as Okuri-gana,3 accompanying characters following Kanji stems in Japanese written words. This is problematic because not all historical texts are manually cor- rected with orthography, and it is time-consuming to annotate by hand. It is one of the major issues in applying NLP tools to Japanese Linguistics be- cause ancient materials often contain a wide vari- ety of orthographic variations. For example, there is an issue of voiced con- sonant marks. Any “Hiragana” character and “Katakana” character (called Kana character alto- gether) represent either consonant (k, s, t, n, h, m, y, r, w) onset with vowel (a, i, u, e, o) nucleus or only the vowel (except for nasal codas N). Further- more, the characters alone can not represent syl- lables beginning with a voiced consonant (g, z, d, b) in current orthography. They are spelled with Kana and a voiced consonant mark (゛) to the up- per right (see Figure 1). However, confusingly, it was not …

Content truncated to 500 words