Using multiple recognition hypotheses to improve speech translation

Using Multiple Recognition Hypotheses to Improve Speech Translation Ruiqiang Zhang, Genichiro Kikui, Hirofumi Yamamoto ATR Spoken Language Translation Research Laboratories 2-2 Hikaridai, Seiika-cho, Soraku-gun, Kyoto, 619-0288, Japan {ruiqiang.zhang, genichiro.kikui, hirofumi.yamamoto}@atr.jp Abstract This paper describes our recent work on integrating speech recognition and machine translation for improving speech translation performance. Two approaches are applied and their performance are evaluated in the workshop of IWSLT 2005. The first is direct N-best hypothesis translation, and the second, a pseudo-lattice decoding algorithm for translating word lattice, can dramatically reduce compu- tation cost incurred by the first approach. We found in the experiments that both of these approaches could improve speech translation significantly. 1. Introduction At least two components are involved in speech to speech translation: automatic speech recognizer and machine trans- lation. Unlike plain text translation, the performance of speech translation may be degraded due to the speech recognition errors. Several approaches have been proposed to compen- sate for the loss of recognition accuracy. [1] proposed N-best recognition hypothesis translation, which trans- lates all the top N hypotheses and then outputs the high- est scored translations by ways of weighing all the trans- lations using a log-linear model. [2] used word lattices to improve translations. [3] used finite state transducers (FST) to convey the features from acoustic analysis and source target translation models. All these approaches realized an integration between speech recognition mod- ules and machine translation modules so that information from speech recognition, such as acoustic model score and language model score, can be exploited in the trans- lation module to achieve the maximum performance over the single-best translation. In the field of machine translation, the phrase-based statistical machine translation approach is widely accepted at present. The related literature can be found in [4] [5]. But previously, word-based statistical machine transla- tion, pioneered by IBM Models 1 to 5 [6], were used widely. In the evaluation, we used both the word-based and phrase-based systems. However, the purpose of this work is not to compare performance of word-based with phrase-based translation. We used two system for dif- ferent translations. The phrase-based SMT is used in X utterance recognized text target translation E best translation J N 1 ASR SMT E NxK 1 Rescore Figure 1: N-best hypothesis translation Chinese-English translation while the word-based SMT is used in Japanese-English translation. In this paper we describe two speech translation struc- tures. The first is a direct N-best hypothesis translation system that uses a text-based machine translation engine to translate each of the hypotheses, and the results are rescored by a log-linear model. The second is a pseudo- lattice translation system, merging the N-best hypotheses into a compact pseudo-lattice which serves as an input to our proposed decoding algorithm for lattice translation. This algorithm runs much faster than the first approach. In the following, Section 2 describes the direct N-best hypothesis translation. Section 3 describes the pseudo- lattice translation. Section 4 introduces the experimental process and translation results in the evaluation of IWSLT2005. Section 5 presents our …

Content truncated to 500 words