Augmenting Translation Models with Simulated Acoustic Confusions for Improved Spoken Language Translation

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 616–625, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics Augmenting Translation Models with Simulated Acoustic Confusions for Improved Spoken Language Translation Yulia Tsvetkov Florian Metze Chris Dyer Language Technologies Institute Carnegie Mellon University Pittsburgh, PA 15213; U.S.A. {ytsvetko, fmetze, cdyer}@cs.cmu.edu Abstract We propose a novel technique for adapting text-based statistical machine translation to deal with input from automatic speech recognition in spoken language translation tasks. We simulate likely misrecognition errors using only a source language pro- nunciation dictionary and language model (i.e., without an acoustic model), and use these to augment the phrase table of a stan- dard MT system. The augmented sys- tem can thus recover from recognition er- rors during decoding using synthesized phrases. Using the outputs of five differ- ent English ASR systems as input, we find consistent and significant improvements in translation quality. Our proposed tech- nique can also be used in conjunction with lattices as ASR output, leading to further improvements. 1 Introduction Spoken language translation (SLT) systems gen- erally consist of two components: (i) an auto- matic speech recognition (ASR) system that tran- scribes source language utterances and (ii) a ma- chine translation (MT) system that translates the transcriptions into the target language. These two components are usually developed independently and then combined and integrated (Ney, 1999; Matusov et al., 2006; Casacuberta et al., 2008; Zhou, 2013; He and Deng, 2013). While this architecture is attractive since it re- lies only on components that are independently useful, such systems face several challenges. First, spoken language tends to be quite different from the highly edited parallel texts that are available to train translation systems. For example, disfluen- cies, such as repeated words or phrases, restarts, and revisions of content, are frequent in spon- taneous speech,1 while these are usually absent in written texts. In addition, ASR outputs typi- cally lack explicit segmentation into sentences, as well as reliable casing and punctuation informa- tion, which are crucial for MT and other text-based language processing applications (Ostendorf et al., 2008). Second, ASR systems are imperfect and make recognition errors. Even high quality sys- tems make recognition errors, especially in acous- tically similar words with similar language model scores, for example morphological substitutions like confusing bare stem and past tense forms, and in high-frequency short words (function words) which often lack both disambiguating context and are subject to reduced pronunciations (Goldwater et al., 2010). One would expect that training an MT system on ASR outputs (rather than the usual written- style texts) would improve matters. Unfortunately, there are few corpora of speech paired with text translations into a second language that could be used for this purpose. This has been an incentive to various MT adaptation approaches and devel- opment of speech-input MT systems. MT adapta- tion has been done via input text pre-processing, by transformation of spoken language (ASR out- put) into written language (MT input) (Peitz et al., …

Content truncated to 500 words