Universal Grapheme-to-Phoneme Prediction Over Latin Alphabets

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 332–343, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Universal Grapheme-to-Phoneme Prediction Over Latin Alphabets Young-Bum Kim and Benjamin Snyder University of Wisconsin-Madison {ybkim,bsnyder}@cs.wisc.edu Abstract We consider the problem of inducing grapheme-to-phoneme mappings for un- known languages written in a Latin alphabet. First, we collect a data-set of 107 languages with known grapheme-phoneme relationships, along with a short text in each language. We then cast our task in the framework of super- vised learning, where each known language serves as a training example, and predictions are made on unknown languages. We induce an undirected graphical model that learns phonotactic regularities, thus relating textual patterns to plausible phonemic interpretations across the entire range of languages. Our model correctly predicts grapheme-phoneme pairs with over 88% F1-measure. 1 Introduction Written language is one of the defining technologies of human civilization, and has been independently invented at least three times through the course of history (Daniels and Bright, 1996). In many ways written language reflects its more primary spoken counterpart. Both are subject to some of the same forces of change, including human migration, cul- tural influence, and imposition by empire. In other ways, written language harkens further to the past, reflecting aspects of languages long since gone from their spoken forms. In this paper, we argue that this imperfect relationship between written symbol and spoken sound can be automatically inferred from textual patterns. By examining data for over 100 languages, we train a statistical model to automat- ically relate graphemic patterns in text to phonemic sequences for never-before-seen languages. We focus here on the the alphabet, a writing sys- tem that has come down to us from the Sumerians. In an idealized alphabetic system, each phoneme in the language is unambiguously represented by a single grapheme. In practice of course, this ideal is never achieved. When existing alphabets are melded onto new languages, they must be imper- fectly adapted to a new sound system. In this paper, we exploit the fact that a single alphabet, that of the Romans, has been adapted to a very large variety of languages. Recent research has demonstrated the effective- ness of cross-lingual analysis. The joint analysis of several languages can increase model accuracy, and enable the development of computational tools for languages with minimal linguistic resources. Previ- ous work has focused on settings where just a hand- ful of languages are available. We treat the task of grapheme-to-phoneme analysis as a test case for larger scale multilingual learning, harnessing infor- mation from dozens of languages. On a more practical note, accurately relating graphemes and phonemes to one another is cru- cial for tasks such as automatic speech recognition and text-to-speech generation. While pronunciation dictionaries and transcribed audio are available for some languages, these resources are entirely lack- ing for the vast majority of the world’s languages. Thus, automatic and generic methods for determin- ing sound-symbol …

Content truncated to 500 words