Epitran: Precision G2P for Many Languages

Epitran: Precision G2P for Many Languages David R. Mortensen, Siddharth Dalmia, Patrick Littell Language Technologies Institute, Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA 15213, USA dmortens@cs.cmu.edu, sdalmia@andrew.cmu.edu, plittell@cs.cmu.edu Abstract Epitran is a massively multilingual, multiple back-end system for G2P (grapheme-to-phoneme) transduction which is distributed with support for 61 languages. It takes word tokens in the orthography of a language and outputs a phonemic representation in either IPA or X-SAMPA. The main system is written in Python and is publicly available as open source software. Its efficacy has been demonstrated in multiple research projects relating to language transfer, polyglot models, and speech. In a particular ASR task, Epitran was shown to improve the word error rate over Babel baselines for acoustic modeling. Keywords: G2P, grapheme, phoneme, International Phonetic Alphabet, IPA, X-SAMPA, multilingual, speech, transfer, polyglot 1. Introduction Epitran is a massively multilingual G2P system. To max- imize its usefulness, it is written in Python and distributed as open source software under an MIT license. Out of the box, it supports 61 languages. Additional languages can easily be added using either a simple, rule-based framework or by adding other back-ends. It has a number of advantages over other G2P and romanization packages like Unitran and URoman including sensible handling of different Latin scripts, precision transduction for each language-script pair (important when multiple languages use the same script dif- ferently), and proper use of international and de facto stan- dards for phonetic representation (IPA, X-SAMPA). 2. Motivation and Related Work A number of Speech and NLP tasks require a G2P step—the conversion of orthographic representations to phonemic or phonetic representations (Daelemans and van den Bosch, 1996; Black et al., 1998; Sproat, 2006; Bisani and Ney, 2008; Laurent et al., 2009; Rao et al., 2015). Such tasks include TTS, ASR, the training of polyglot phonetic mod- els for (non-speech) NLP tasks, and the implementation of approximate phonetic matching. G2P research has focused on some of the most difficult cases, including English. Less attention has been paid to providing G2P coverage for the wide range of languages with a more direct relationship be- tween orthography and phonology. Many such languages require more than simple a mapping table for G2P but can be handled adequately by a more sophisticated rule-based system and certainly do not require a machine learning ap- proach. This is fortunate, because many of these are also low resource languages in which sufficient training data, even for training a WFST, are not available but adequate linguistic descriptions do exist. There are a few tools that already occupy this niche. Uni- tran is a tool that converts orthographic text to WorldBet and X-SAMPA (Qian et al., 2010). It is limited, though, in that it does not support Roman scripts and it does not have a mechanism for specifying different behavior for dif- ferent languages that use the same script. For example, Amharic and Tigrinya both use the Ethiopic script and are treated exactly the same by Unitran even though their use of the …

Content truncated to 500 words