Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languages

Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languages Sonal Sannigrahi Saarland University, Saarland Informatics Campus Saarbrücken, Germany sosa00001@stud.uni-saarland.de Rachel Bawden Inria, Paris, France rachel.bawden@inria.fr Abstract Multilingual language models have shown impressive cross-lingual transfer ability across a diverse set of languages and tasks. To improve the cross-lingual ability of these models, some strategies include transliter- ation and finer-grained segmentation into characters as opposed to subwords. In this work, we investigate lexical sharing in multilingual machine translation (MT) from Hindi, Gujarati, Nepali into English. We explore the trade-offs that exist in translation performance between data sam- pling and vocabulary size, and we explore whether transliteration is useful in encour- aging cross-script generalisation. We also verify how the different settings generalise to unseen languages (Marathi and Bengali). We find that transliteration does not give pronounced improvements and our analysis suggests that our multilingual MT models trained on original scripts seem to already be robust to cross-script differences even for relatively low-resource languages. Our code will be made publicly available.1 1 Introduction As research in natural language processing (NLP) moves towards handling an increasing number of languages (Aharoni et al., 2019; Fan et al., 2021), one of the key challenges is targeting low-resource and morphologically rich languages (Johnson et al., 2017; Magueresse et al., 2020). Multilingual © 2023 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. 1https://github.com/sonalsannigrahi/ Multilingual_Strategy language models such as mBERT (Devlin et al., 2019) and XLM-R (Conneau et al., 2020) have shown surprising cross-lingual ability in zero and few-shot scenarios for a diverse set of languages (Wu and Dredze, 2020). In order for low-resource languages to optimally benefit from data available for related and higher- resource languages, one research direction has been to explore what encourages better cross-lingual sharing in multilingual models, particularly in mod- els that have joint vocabularies (Ha et al., 2016; Johnson et al., 2017; Aharoni et al., 2019). One strategy for doing this is to preprocess the texts to reduce variation linked to differences in script and orthographic conventions, for example phonetisation, transliteration and transcription, in order to increase lexical overlap across languages. These pre-processing steps have been used in the literature across several multilingual NLP tasks (Nakov and Tiedemann, 2012; Nguyen and Chi- ang, 2017; Chakravarthi et al., 2019; Goyal et al., 2020; Sun et al., 2022; Muller et al., 2021; Alabi et al., 2022). However, there is still some debate over how much transliteration helps in multilingual setups, despite it theoretically encouraging better lexical overlap, particularly for low-resource lan- guages. For example, Pires et al. (2019) found that transfer may be helped by increased lexical over- lap (although it also works without it) and K et al. (2020) argue that lexical overlap has a negligible impact on transfer. Chakravarthi et al. (2019) and Muller et al. (2021) found gains when transliterat- ing, whereas for Alabi et al. (2022), results were less clear. In this study, we build on this previous work to further …

Content truncated to 500 words