A Large-Vocabulary Bilingual Speech Recognition System for Chinese and Japanese Language

A Large-Vocabulary Bilingual Speech Recognition System for Chinese and Japanese Language Jyh-Shing Shyuu and Jhing-Fa Wang Department of Computer Science and Information Engineering National Cheng Kung University 1, Ta-Hsueh Road, Tainan, Taiwan, R.O.C. wangjf@server2.iie.ncku.edu.tw ABSTRACT Bilingual or Multilingual speech recognition gradually becomes an attractive research topic because bilingual writings appear almost everywhere in present day. In this paper, we propose a continuous word-based speech recognition system to dictate the Mandarin and Japanese speech simultaneously. We find that there are about 62 basic phoneme like units(PLUs) among the mixed Mandarin and Japanese syllables. The 62 HA/Ms are used to decode the input speech into word hypotheses based on a fast tree-beam searching algorithm. In the language model, the bigram model and trigram model are used to select the most likely word from the word candidates. We also have a bilingual dictionary to deal with the cross language information. Our proposed system architecture can not only dictate Mandarin and Japanese speech simultaneously but also provide a possible solution to recognize any other bilingual speech. I. INTRODUCTION Bilingual or Multilingual speech recognition gradually becomes an attractive research topic because bilingual writings appear almost everywhere in present day. In this paper, a continuous word-based speech recognition system is proposed for dictating the Mandarin and Japanese speech simultaneously. In the acoustic processing model, we segment Mandarin syllables into phoneme like units (PLUs) instead of using traditional 21 consonants and 38 vowels in recognition. We find that there are totally 62 basic PLUs among the mixed Mandarin and Japanese syllables[1,2]. By using the PLUs, it can not only increase the recognition speed but also increase the size of training patterns for each recognition unit. Because Chinese is a tonal language, tone recognition is also included in our recognition system by a proposed concatenated-tone recognition model. In the language processing model, two sets of bigram model and trigram model were used to dictate Japanese and Chinese language respectively. For the cross language information, if the cross language information is not available, a bilingual dictionary is used to translate the Japanese to Chinese or Chinese to Japanese. Hence, the bigram and trigram information can be obtained from the Chinese or Japanese language model respectively. There is also another alternative solution to deal with bilingual trigram and bigram information. We can assume that the bigram and trigram information are equal for transitions from Chinese to Japanese or from Japanese to Chinese. The assumption is based on the observation that most of the transitions are caused by the noun. The assumption can be eliminated or replaced when the bilingual information can be estimated from a bilingual text corpus. In the acoustic training phase, 400 Japanese balanced-sentences and 320 Chinese balanced-sentences were collected for training the 62 acoustic HIVIMs. The balanced-sentences are automatically generated by a previously proposed balanced-corpus generating algorithm[3]. In the recognition phase, a Viterbi tree-beam search algorithm is adopted to obtain the top-N most likely word candidates in frame synchronously[5]. By combining the acoustic scores with the language …

Content truncated to 500 words