Massively Multilingual Adversarial Speech Recognition

Proceedings of NAACL-HLT 2019, pages 96–108 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 96 Massively Multilingual Adversarial Speech Recognition Oliver Adams, Matthew Wiesner, Shinji Watanabe, and David Yarowsky Department of Computer Science Johns Hopkins University, Baltimore MD, USA {oadams1,wiesner,shinjiw,yarowsky}@jhu.edu Abstract We report on adaptation of multilingual end- to-end speech recognition models trained on as many as 100 languages. Our findings shed light on the relative importance of similarity between the target and pretraining languages along the dimensions of phonetics, phonol- ogy, language family, geographical location, and orthography. In this context, experi- ments demonstrate the effectiveness of two additional pretraining objectives in encourag- ing language-independent encoder representa- tions: a context-independent phoneme objec- tive paired with a language-adversarial classi- fication objective. 1 Introduction The main difficulty in creating automatic speech recognition (ASR) systems for a large number of the world’s 7,000 languages is a lack of training data. Such data comes in the form of speech paired with transcriptions, a pronunciation lexicon, and text for language model training. A common technique in data-constrained settings is to learn language-independent representations of speech via multilingual training. Popular approaches in- clude the use of multilingual bottleneck features (Vesely et al., 2012) as well as multilingual model training before fine-tuning to a target language (Scanzio et al., 2008; Vu et al., 2012). Prior work in multilingual and cross-lingual speech recognition has been restricted to a small handful of the world’s most-spoken languages, relying on multilingual corpora such as Global- Phone (Schultz, 2002), the IARPA Babel cor- pora (Gales et al., 2014), or the VoxForge1 cor- pora. Most work typically only reports on models trained on a subset of these languages. In this paper we explore pretraining multilin- gual ASR models using speech from as many as 1voxforge.org 100 languages from the CMU Wilderness Mul- tilingual Speech Dataset (Black, 2019).2 To the best of our knowledge, this is the greatest num- ber of languages that has been used in multilingual ASR model training to date. We perform experi- ments to guide the choice of languages used when pretraining the model and assess the relative im- portance of similarity between the pretraining lan- guages and target language in terms of geographic location, phonology, phonetic inventory, language family and orthography. We examine these variables in the context of two experimental setups: one where models are adapted to target language and target speakers, and one where models are adapted to target lan- guage but non-target speakers. The first task is relevant to language documentation contexts, which often involves transcribing speech of spe- cific speakers for which there already exists some transcribed speech as training data (Michaud et al., 2018). The second case is relevant to incident re- sponse as modelled by LORELEI (Strassel and Tracey, 2016), where there may only be a sin- gle target-language consultant available for which transcribed speech can be elicited, but the goal is to have an ASR model that generalizes to multiple speakers. Multilingual ASR training on such a …

Content truncated to 500 words