Segmentation Strategies for Streaming Speech Translation

Proceedings of NAACL-HLT 2013, pages 230–238, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Segmentation Strategies for Streaming Speech Translation Vivek Kumar Rangarajan Sridhar, John Chen, Srinivas Bangalore Andrej Ljolje, Rathinavelu Chengalvarayan AT&T Labs - Research 180 Park Avenue, Florham Park, NJ 07932 vkumar,jchen,srini,alj,rathi@research.att.com Abstract The study presented in this work is a first ef- fort at real-time speech translation of TED talks, a compendium of public talks with dif- ferent speakers addressing a variety of top- ics. We address the goal of achieving a sys- tem that balances translation accuracy and la- tency. In order to improve ASR performance for our diverse data set, adaptation techniques such as constrained model adaptation and vo- cal tract length normalization are found to be useful. In order to improve machine transla- tion (MT) performance, techniques that could be employed in real-time such as monotonic and partial translation retention are found to be of use. We also experiment with inserting text segmenters of various types between ASR and MT in a series of real-time translation ex- periments. Among other results, our experi- ments demonstrate that a good segmentation is useful, and a novel conjunction-based seg- mentation strategy improves translation qual- ity nearly as much as other strategies such as comma-based segmentation. It was also found to be important to synchronize various pipeline components in order to minimize la- tency. 1 Introduction The quality of automatic speech-to-text and speech- to-speech (S2S) translation has improved so signifi- cantly over the last several decades that such systems are now widely deployed and used by an increasing number of consumers. Under the hood, the individ- ual components such as automatic speech recogni- tion (ASR), machine translation (MT) and text-to- speech synthesis (TTS) that constitute a S2S sys- tem are still loosely coupled and typically trained on disparate data and domains. Nevertheless, the models as well as the pipeline have been optimized in several ways to achieve tasks such as high qual- ity offline speech translation (Cohen, 2007; Kings- bury et al., 2011; Federico et al., 2011), on-demand web based speech and text translation, low-latency real-time translation (Wahlster, 2000; Hamon et al., 2009; Bangalore et al., 2012), etc. The design of a S2S translation system is highly dependent on the nature of the audio stimuli. For example, talks, lec- tures and audio broadcasts are typically long and re- quire appropriate segmentation strategies to chunk the input signal to ensure high quality translation. In contrast, single utterance translation in several consumer applications (apps) are typically short and can be processed without the need for additional chunking. Another key parameter in designing a S2S translation system for any task is latency. In offline scenarios where high latencies are permit- ted, several adaptation strategies (speaker, language model, translation model), denser data structures (N- best lists, word sausages, lattices) and rescoring pro- cedures can be utilized to improve the quality of end-to-end translation. On the other hand, real- time speech-to-text or speech-to-speech translation demand the best possible accuracy at low …

Content truncated to 500 words