PronouncUR: An Urdu Pronunciation Lexicon Generator

PronouncUR: An Urdu Pronunciation Lexicon Generator Haris Bin Zia1, Agha Ali Raza1, Awais Athar2 1Information Technology University, 6th Floor, Arfa Software Technology Park, Ferozepur Road, Lahore, Pakistan 2EMBL-EBI, Wellcome Genome Campus, Hinxton, Cambridgeshire, CB10 1SD, UK {haris.zia, agha.ali.raza}@itu.edu.pk awais@ebi.ac.uk Abstract State-of-the-art speech recognition systems rely heavily on three basic components: an acoustic model, a pronunciation lexicon and a language model. To build these components, a researcher needs linguistic as well as technical expertise, which is a barrier in low- resource domains. Techniques to construct these three components without having expert domain knowledge are in great demand. Urdu, despite having millions of speakers all over the world, is a low-resource language in terms of standard publically available linguistic resources. In this paper, we present a grapheme-to-phoneme conversion tool for Urdu that generates a pronunciation lexicon in a form suitable for use with speech recognition systems from a list of Urdu words. The tool predicts the pronunciation of words using a LSTM-based model trained on a handcrafted expert lexicon of around 39,000 words and shows an accuracy of 64% upon internal evaluation. For external evaluation on a speech recognition task, we obtain a word error rate comparable to one achieved using a fully handcrafted expert lexicon. Keywords: Pronunciation Lexicon, Pronunciation Modeling, Lexicon Learning, Speech Recognition, Urdu 1. Introduction Automatic Speech Recognition (ASR) for resource scarce languages has been an active research area in the past few years (Sherwani, 2009; Qiao, 2010; Chan, 2012). Modern speech recognition systems usually require three resources: transcribed speech for acoustic modeling, a large text data for language modeling and a pronunciation lexicon that maps words to sub-word units known as phonemes. Pronunciation lexicon acts as a link connecting language model with the acoustic model. While it is comparatively easy to gather transcribed speech waveforms and large text datasets, developing a pronunciation dictionary is quite expensive and requires tremendous amount of manual effort and linguistic expertise. Therefore, development of a pronunciation lexicon is the bottleneck when building ASR systems for low-resource languages. Techniques to reduce the need of expert knowledge in design and development of pronunciation lexicons are in great demand. We are interested in developing a pronunciation lexicon generation tool for Urdu which is an Indo-Aryan language spoken widely with over 100 million speakers1. Urdu is official language of Pakistan. Its writing system is Segmental and more specifically Abjad i.e. only consonants are marked while vowels (diacritics) are optional. Urdu follows Arabic script written from right to left. A sentence written in Urdu along with its English translation is given below: اودر ۔ ےہ نابز یموق یک ناتسکاپ Urdu is the national language of Pakistan. Automatic Speech Recognition (ASR) research for Urdu exhibits number of challenges which are discussed in detail in subsequent sections. Despite being spoken by millions of speakers all over the world, Urdu is low- resource in terms of standard publically available linguistic resources. 1 https://www.ethnologue.com/language/urd To our best knowledge, our Urdu pronunciation lexicon generation tool is the first tool of its kind that …

Content truncated to 500 words