An Attentional Model for Speech Translation Without Transcription

Proceedings of NAACL-HLT 2016, pages 949–959, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics An Attentional Model for Speech Translation Without Transcription Long Duong,12 Antonios Anastasopoulos,3 David Chiang,3 Steven Bird14 and Trevor Cohn1 1Department of Computing and Information Systems, University of Melbourne 2National ICT Australia, Victoria Research Laboratory 3Department of Computer Science and Engineering, University of Notre Dame 4International Computer Science Institute, University of California Berkeley Abstract For many low-resource languages, spoken lan- guage resources are more likely to be an- notated with translations than transcriptions. This bilingual speech data can be used for word-spotting, spoken document retrieval, and even for documentation of endangered lan- guages. We experiment with the neural, atten- tional model applied to this data. On phone- to-word alignment and translation reranking tasks, we achieve large improvements relative to several baselines. On the more challeng- ing speech-to-word alignment task, our model nearly matches GIZA++’s performance on gold transcriptions, but without recourse to transcriptions or to a lexicon. 1 Introduction For many low-resource languages, spoken language resources are more likely to come with translations than with transcriptions. Most of the world’s lan- guages are not written, so there is no orthography for transcription. Phonetic transcription is possible but too costly to produce at scale. Even when a mi- nority language has an official orthography, people are often only literate in the language of formal ed- ucation, such as the national language. Neverthe- less, it is relatively easy to provide written or spoken translations for audio sources. Subtitled or dubbed movies are a widespread example. One application of models of bilingual speech data is documentation of endangered languages. Since most speakers are bilingual in a higher- resource language, they can listen to a source lan- guage recording sentence by sentence and provide a spoken translation (Bird, 2010; Bird et al., 2014). By aligning this data at the word level, we hope to automatically identify regions of data where further evidence is needed, leading to a substantial, inter- pretable record of the language that can be studied even if the language falls out of use (Abney and Bird, 2010; Bird and Chiang, 2012). We experiment with extensions of the neural, at- tentional model of Bahdanau et al. (2015), work- ing at the phone level or directly on the speech sig- nal. We assume that the target language is a high- resource language such as English that can be auto- matically transcribed; therefore, in our experiments, the target side is text rather than the output of an au- tomatic speech recognition (ASR) system. In the first set of experiments, as a stepping stone to direct modeling of speech, we represent the source as a sequence of phones. For phone-to- word alignment, we obtain improvements of 9–24% absolute F1 over several baselines (Och and Ney, 2000; Neubig et al., 2011; Stahlberg et al., 2012). For phone-to-word translation, we use our model to rerank n-best lists from Moses (Koehn et al., 2007) and observe improvements in BLEU of 0.9–1.7. In …

Content truncated to 500 words