Training Data Augmentation for Code-Mixed Translation

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5760–5766 June 6–11, 2021. ©2021 Association for Computational Linguistics 5760 Training Data Augmentation for Code-Mixed Translation Abhirut Gupta Google Research abhirut@google.com Aditya Vavre and Sunita Sarawagi IIT Bombay {adityavavre,sunita}@cse.iitb.ac.in Abstract Machine translation of user-generated code- mixed inputs to English is of crucial impor- tance in applications like web search and tar- geted advertising. We address the scarcity of parallel training data for training such models by designing a strategy of converting existing non-code-mixed parallel data sources to code- mixed parallel data. We present an mBERT based procedure whose core learnable com- ponent is a ternary sequence labeling model, that can be trained with a limited code-mixed corpus alone. We show a 5.8 point increase in BLEU on heavily code-mixed sentences by training a translation model using our data aug- mentation strategy on an Hindi-English code- mixed translation task. 1 Introduction Code-mixing (CM), the phenomenon of mixing words from two languages in a sentence, is get- ting increasingly commonplace in several bilin- gual communities1. Recently, much research has focused on training language models over code- switched data for tasks like automatic speech recog- nition (ASR) (Winata et al., 2019; Gonen and Gold- berg, 2019). In this paper we focus on the less ex- plored problem of translating code-switched inputs to a high-resource language like English. This task is compelling in applications like Web search, tar- geted advertising, and recommendations, which re- quire matching user-generated code-mixed queries to rich English content. A major challenge in such applications is the lack of parallel data from code-mixed input to English. While years of effort have made avail- able rich parallel datasets for translation, these are mostly over formal sources like news, which tend to be less code-mixed. In this paper we show how to create high-quality parallel data for training a code-mixed translation model by exploiting three 1https://github.com/gentaiscool/code-switching-papers types of resources: 1) Parallel data from non-code- mixed sentences to English, 2) Code-mixed sen- tences, and 3) Monolingual sentences in English. Contributions: (1) We present an mBERT (De- vlin et al., 2019) based procedure for converting non-CM parallel data to CM parallel data. The core learnable component of our procedure requires fine-tuning mBERT for a three-way sequence la- beling task, and can be easily trained using the limited code-switched sentences alone. We apply this model to convert source sentences of the par- allel data to code-mixed sentences, while keeping the target English sentences in-tact. We also ex- tend the existing back-translation method of using monolingual target data, with our code-switched augmentation. (2) We experiment on a rich public code-mixed dataset obtained from a literacy promo- tion project. We show that with our data augmenta- tion strategy the translation BLEU improves from 43.9 to 46.4 overall. On sentences that are more heavily code-mixed our accuracy increases by 5.8 BLEU points, and on an adversarial test set where the baseline provides poor accuracy we show a 5.4 …

Content truncated to 500 words