Japanese Pronunciation Prediction as Phrasal Statistical Machine Translation

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 120–128, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Japanese Pronunciation Prediction as Phrasal Statistical Machine Translation Jun Hatori1 Hisami Suzuki2 1Department of Computer Science, University of Tokyo 7-3-1 Hongo, Bunkyo, Tokyo 113-0033, Japan 2Microsoft Research / One Microsoft Way, Redmond, WA 98052, USA hatori@is.s.u-tokyo.ac.jp hisamis@microsoft.com Abstract This paper addresses the problem of pre- dicting the pronunciation of Japanese text. The difficulty of this task lies in the high degree of ambiguity in the pronunciation of Japanese characters and words. Previ- ous approaches have either considered the task as a word-level classification problem based on a dictionary, which does not fare well in handling out-of-vocabulary (OOV) words; or solely focused on the pronun- ciation prediction of OOV words without considering the contextual disambiguation of word pronunciations in text. In this paper, we propose a unified approach within the framework of phrasal statisti- cal machine translation (SMT) that com- bines the strengths of the dictionary-based and substring-based approaches. Our ap- proach is novel in that we combine word- and character-based pronunciations from a dictionary within an SMT framework: the former captures the idiosyncratic proper- ties of word pronunciation, while the latter provides the flexibility to predict the pro- nunciation of OOV words. We show that based on an extensive evaluation on vari- ous test sets, our model significantly out- performs the previous state-of-the-art sys- tems, achieving around 90% accuracy in most domains. 1 Introduction This paper1 explores the problem of assigning pro- nunciation to Japanese text, which consists of a mixture of ideographic and phonetic characters. The task is naturally important for the text-to- speech application (Schroeter et al., 2002), and has been researched in that context as letter-to- phoneme conversion, which converts an ortho- 1This work was conducted during the first author’s intern- ship at Microsoft Research. graphic character sequence into phonemes. In addition to speech applications, the task is also crucial for those languages such as Chinese and Japanese, where users generally type in the pro- nunciations of words, which are then converted into the desired character string via the software application called input methods (e.g. Gao et al. (2002a); Gao et al. (2002b)). Predicting the pronunciation of Japanese text is particularly challenging because the word and character pronunciations are highly ambiguous. Japanese orthography employs four sets of char- acters: hiragana and katakana (called generally as kana), which are syllabary systems and thus phonemic; kanji, which is ideographic and con- sists of several thousand characters; and Roman alphabet. Out of these, kanji characters typi- cally have multiple possible pronunciations2; es- pecially those in frequent use tend to have many — between 5 and 10, sometimes as many as 20. This yields an exponential number of pronunci- ation possibilities when multiple kanji characters are combined in a word. Also, the pronunciation of a word is frequently idiosyncratic. This idiosyncratic property of the word pronun- ciation naturally motivates us to take a dictionary- based approach. Traditionally, most …

Content truncated to 500 words