Leveraging Orthographic Similarity for Multilingual Neural Transliteration

Leveraging Orthographic Similarity for Multilingual Neural Transliteration Anoop Kunchukuttan1, Mitesh Khapra2, Gurneet Singh2, Pushpak Bhattacharyya1 {anoopk,pb}@cse.iitb.ac.in, {miteshk,garry}@cse.iitm.ac.in 1Department of Computer Science & Engineering Indian Institute of Technology Bombay, Mumbai, India. 2Department of Computer Science & Engineering Indian Institute of Technology Madras, Chennai, India. Abstract We address the task of joint training of translit- eration models for multiple language pairs (multilingual transliteration). This is an in- stance of multitask learning, where individ- ual tasks (language pairs) benefit from shar- ing knowledge with related tasks. We fo- cus on transliteration involving related tasks i.e., languages sharing writing systems and phonetic properties (orthographically similar languages). We propose a modified neural encoder-decoder model that maximizes pa- rameter sharing across language pairs in order to effectively leverage orthographic similar- ity. We show that multilingual transliteration significantly outperforms bilingual translitera- tion in different scenarios (average increase of 58% across a variety of languages we experi- mented with). We also show that multilingual transliteration models can generalize well to languages/language pairs not encountered dur- ing training and hence perform well on the ze- roshot transliteration task. We show that fur- ther improvements can be achieved by using phonetic feature input. 1 Introduction Transliteration is a key building block for multi- lingual and cross-lingual NLP since it is essential for (i) handling of names in applications like ma- chine translation (MT) and cross-lingual information retrieval (CLIR), and (ii) user-friendly input meth- ods. The transliteration problem has been exten- sively studied in literature for a variety of language pairs (Karimi et al., 2011). Previous work has looked at the most natural setup - training on a single lan- guage pair. However, no prior work exists on jointly training multiple language pairs (referred to as mul- tilingual transliteration henceforth). Multilingual transliteration can be seen as an in- stance of multi-task learning, where training each language pair constitutes a task. Multi-task learning works best when the tasks are related to each other, so sharing of knowledge across tasks is beneficial. Thus, multilingual transliteration can be beneficial, if the languages involved are related. We identify such a natural and practically useful scenario: mul- tilingual transliteration involving languages that are related on account of sharing writing systems and phonetic properties. We refer to such languages as orthographically similar languages. We say that two languages are orthographically similar if they have: (i) highly overlapping phoneme sets, (ii) mutually compatible orthographic systems, and (iii) similar grapheme to phoneme mappings. For instance, Indo-Aryan languages largely share the same set of phonemes. They use different In- dic scripts, but correspondences can be established between equivalent characters across scripts. For example, the Hindi (Devanagari script) character क (ka) maps to the Bengali ক (ka) which stands for the consonant sound (IPA: k). The grapheme to phoneme mapping is also consistent for equiva- lent characters. We can identify two major sources of orthographic similarity: (a) genetic relationship between languages (groups like Romance, Slavic, Indo-Aryan and Turkic languages) (b) prolonged contact between languages over a long period of time, e.g. …

Content truncated to 500 words