The University of Edinburgh’s Submission to the WMT22 Code-Mixing Shared Task (MixMT)

Proceedings of the Seventh Conference on Machine Translation (WMT), pages 1145 - 1157 December 7-8, 2022 ©2022 Association for Computational Linguistics The University of Edinburgh’s Submission to the WMT22 Code-Mixing Shared Task (MixMT) Faheem Kirefu Vivek Iyer Pinzhen Chen Laurie Burchell School of Informatics, University of Edinburgh {fkirefu,vivek.iyer,pinzhen.chen,laurie.burchell}@ed.ac.uk Abstract The University of Edinburgh participated in the WMT22 shared task on code-mixed trans- lation. This consists of two subtasks: i) gen- erating code-mixed Hindi/English (Hinglish) text generation from parallel Hindi and En- glish sentences and ii) machine translation from Hinglish to English. As both subtasks are con- sidered low-resource, we focused our efforts on careful data generation and curation, especially the use of backtranslation from monolingual re- sources. For subtask 1 we explored the effects of constrained decoding on English and translit- erated subwords in order to produce Hinglish. For subtask 2, we investigated different pre- training techniques, namely comparing simple initialisation from existing machine translation models and aligned augmentation. For both subtasks, we found that our baseline systems worked best. Our systems for both subtasks were one of the overall top-performing submis- sions. 1 Introduction Code-mixing is the shift from one language to another within a single conversation or utterance (Sitaram et al., 2019). It is an extremely common and diverse communicative phenomenon world- wide (Do˘gruöz et al., 2021; Sitaram et al., 2019), though one which is currently under-served by many NLP technologies (Solorio et al., 2021). One of the most well-known examples of code- mixing is between Hindi and English, commonly referred to as Hinglish1. It is extremely common amongst Hindi-English bilingual speakers in both speech and text, used across a range of genres and media (Parshad et al., 2016), and has its own dis- tinctive features and linguistic forms (Kumar, 1986; Sailaja, 2011). The process of generating Hinglish from the written text is non-trivial, as code-mixing 1In the scope of this paper, we designate “hg” as the lan- guage code for Hinglish. may happen at the phrase or word level, but Hindi and English differ substantially syntactically. As a novel addition to the current code-mixing NLP research, we investigated lexically constrain- ing the Hinglish output in subtask 1 to only contain words from English and Hindi sources. Through analysis, we demonstrated that transliteration mis- matches could affect performance. Another novel approach we explore for this task, particularly for subtask 2, is a denoising-based pre- training technique called Aligned Augmentation (AA) (Pan et al., 2021). AA, which trains MT models to denoise artificially generated code-mixed text, was shown by Pan et al. (2021) to boost trans- lation performance across a variety of languages - thanks to the enhanced transfer learning brought about by code-mixed pretraining. In this work, we explored if this general-purpose approach could be useful for translating authentic, human-generated code-mixed text, focusing on Hinglish. Despite these efforts, we found that for both sub- tasks our original baselines worked better and con- stituted our final submissions for this task, which ranked as one of the top-performing systems for …

Content truncated to 500 words