A Unified Approach in Speech-to-Speech Translation: Integrating Features of Speech recognition and Machine Translation

A Unified Approach in Speech-to-Speech Translation: Integrating Features of Speech Recognition and Machine Translation Ruiqiang Zhang and Genichiro Kikui and Hirofumi Yamamoto Taro Watanabe and Frank Soong and Wai Kit Lo ATR Spoken Language Translation Research Laboratories 2-2 Hikaridai, Seiika-cho, Soraku-gun, Kyoto, 619-0288, Japan {ruiqiang.zhang, genichiro.kikui}@atr.jp Abstract Based upon a statistically trained speech translation system, in this study, we try to combine distinctive features derived from the two modules: speech recognition and statistical machine translation, in a log- linear model. The translation hypotheses are then rescored and translation perfor- mance is improved. The standard trans- lation evaluation metrics, including BLEU, NIST, multiple reference word error rate and its position independent counterpart, were optimized to solve the weights of the features in the log-linear model. The exper- imental results have shown significant im- provement over the baseline IBM model 4 in all automatic translation evaluation met- rics. The largest was for BLEU, by 7.9% absolute. 1 Introduction Current translation systems are typically of a cascaded structure: speech recognition followed by machine translation. This structure, while explicit, lacks some joint optimality in per- formance since the speech recognition module and translation module are running rather in- dependently. Moreover, the translation module of a speech translation system, a natural off- spring of text-input based translation system, usually takes a single-best recognition hypoth- esis transcribed in text and performs standard text-based translation. Lots of supplementary information available from speech recognition, such as N-best recognition recognition hypothe- ses, likelihoods of acoustic and language models, is not well utilized in the translation process. The information can be effective for improving translation quality if employed properly. The supplementary information can be ex- ploited by a tight coupling of speech recognition and machine translation (Ney, 1999) or keeping the cascaded structure unchanged but using an integration model, log-linear model, to rescore the translation hypotheses. In this study the last approach was used due to its explicitness. In this paper we intended to improve speech translation by exploiting these information. Moreover, a number of advanced features from the machine translation module were also added in the models. All the features from the speech recognition and machine translation module were combined by the log-linear models seam- lessly. In order to test our results broadly, we used four automatic translation evaluation metrics: BLEU, NIST, multiple word error rate and po- sition independent word error rate, to measure the translation improvement. In the following, in section 2 we introduce the speech translation system. In section 3, we de- scribe the optimization algorithm used to find the weight parameters in the log-linear model. In section 4 we demonstrate the effectiveness of our technique in speech translation experi- ments. In the final two sections we discuss the results and present our conclusions. 2 Feature-based Log-linear Models in Speech Translation The speech translation experimental system used in this study illustrated in Fig. 1 is a typi- cal, statistics-based one. It consists of two ma- jor cascaded components: an automatic speech recognition (ASR) module and a …

Content truncated to 500 words