A Coarse-Grained Model for Optimal Coupling of ASR and SMT Systems for Speech Translation

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1902–1907, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. A Coarse-Grained Model for Optimal Coupling of ASR and SMT Systems for Speech Translation Gaurav Kumar 1, Graeme Blackwood 2, Jan Trmal 1, Daniel Povey 1, Sanjeev Khudanpur 1 1 CLSP & HLTCOE, Johns Hopkins University, Baltimore, MD, USA 2IBM T. J. Watson Research Center, Yorktown Heights, NY, USA {gkumar6, dpovey1, khudanpur}@jhu.edu, blackwood@us.ibm.com Abstract Speech translation is conventionally car- ried out by cascading an automatic speech recognition (ASR) and a statistical ma- chine translation (SMT) system. The hy- potheses chosen for translation are based on the ASR system’s acoustic and lan- guage model scores, and typically opti- mized for word error rate, ignoring the in- tended downstream use: automatic trans- lation. In this paper, we present a coarse- to-fine model that uses features from the ASR and SMT systems to optimize this coupling. We demonstrate that several standard features utilized by ASR and SMT systems can be used in such a model at the speech-translation interface, and we provide empirical results on the Fisher Spanish-English speech translation cor- pus. 1 Introduction Speech translation is the process of translating speech in the source language to text or speech in the target language. This process is typically structured as a three step pipeline. Step one in- volves training an Automatic Speech Recognition (ASR) system to transcribe speech to text in the source language. Step two involves extracting an appropriate form of the ASR output to translate. We will refer to this step as the Speech-Translation interface. In the simplest scenario, the ASR 1- best output can be used as the source text to trans- late. It may be useful to consider alternative ASR hypotheses and these take the form of an N-best list or a word-lattice. An N-best list can be in- cluded easily into the tuning and the decoding pro- cess of a statistical machine translation (SMT) sys- tem (Zhang et al., 2004). Several researchers have proposed solutions to incorporating lattices and confusion networks in this process (Saleem et al., 2004; Matusov et al., 2005; Bangalore and Ric- cardi, 2000; Dyer et al., 2008a; Bertoldi and Fed- erico, 2005; Quan et al., 2005; Mathias and Byrne, 2006; Bertoldi et al., 2007). Word lattice input to SMT for tuning and decoding increases the com- plexity of the decoding process because of the ex- ponential number of alternatives that are present. Finally, step three involves training and tuning a Statistical Machine Translation (SMT) system and decoding the output extracted through the speech translation interface. This paper presents a featurized model which performs the job of hypothesis selection from the outputs of the ASR system for the input to the SMT system. Our motivation is as follows: 1. Using downstream information : Hypoth- esis selection for the input to the SMT sys- tem should be done jointly by the ASR and the SMT systems. That is, there may exist hypotheses …

Content truncated to 500 words