Grapheme-to-Phoneme Models for (Almost) Any Language

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 399–408, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Grapheme-to-Phoneme Models for (Almost) Any Language Aliya Deri and Kevin Knight Information Sciences Institute Department of Computer Science University of Southern California {aderi, knight}@isi.edu Abstract Grapheme-to-phoneme (g2p) models are rarely available in low-resource languages, as the creation of training and evaluation data is expensive and time-consuming. We use Wiktionary to obtain more than 650k word-pronunciation pairs in more than 500 languages. We then develop phoneme and language distance metrics based on phono- logical and linguistic knowledge; apply- ing those, we adapt g2p models for high- resource languages to create models for related low-resource languages. We pro- vide results for models for 229 adapted lan- guages. 1 Introduction Grapheme-to-phoneme (g2p) models convert words into pronunciations, and are ubiquitous in speech- and text-processing systems. Due to the diversity of scripts, phoneme inventories, phono- tactic constraints, and spelling conventions among the world’s languages, they are typically language- specific. Thus, while most statistical g2p learning methods are language-agnostic, they are trained on language-specific data—namely, a pronunciation dictionary consisting of word-pronunciation pairs, as in Table 1. Building such a dictionary for a new language is both time-consuming and expensive, because it re- quires expertise in both the language and a notation system like the International Phonetic Alphabet, applied to thousands of word-pronunciation pairs. Unsurprisingly, resources have been allocated only to the most heavily-researched languages. Global- Phone, one of the most extensive multilingual text and speech databases, has pronunciation dictionar- ies in only 20 languages (Schultz et al., 2013)1. 1We have been unable to obtain this dataset. lang word pronunciation eng anybody e̞ n iː b ɒ d iː pol żołądka z̻ o w o n̪ t̪ k a ben শক্ s̪ ɔ k t̪ ɔ hebתומולח ʁ a l o m o t Table 1: Examples of English, Polish, Bengali, and Hebrew pronunciation dictionary entries, with pronunciations represented with the International Phonetic Alphabet (IPA). word eng deu nld gift ɡ ɪ f tʰ ɡ ɪ f t ɣ ɪ f t class kʰ l æ s k l aː s k l ɑ s send s e̞ n d z ɛ n t s ɛ n t Table 2: Example pronunciations of English words using English, German, and Dutch g2p models. For most of the world’s more than 7,100 lan- guages (Lewis et al., 2009), no data exists and the many technologies enabled by g2p models are in- accessible. Intuitively, however, pronouncing an unknown language should not necessarily require large amounts of language-specific knowledge or data. A native German or Dutch speaker, with no knowl- edge of English, can approximate the pronuncia- tions of an English word, albeit with slightly differ- ent phonemes. Table 2 demonstrates that German and Dutch g2p models can do the same. Motivated by this, we create and evaluate g2p models for low-resource languages by adapting ex- isting g2p models for high-resource languages us- ing linguistic and …

Content truncated to 500 words