CoCoa: An Encoder-Decoder Model for Controllable Code-switched Generation

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2466 - 2479 December 7-11, 2022 ©2022 Association for Computational Linguistics COCOA: An Encoder-Decoder Model for Controllable Code-switched Generation Sneha Mondal1, Ritika.2∗, Shreya Pathak2∗, Preethi Jyothi2, Aravindan Raghuveer1 1Google Research 2IIT Bombay {snehamondal, araghuveer}@google.com, {ritikagoyal, shreyapathak, pjyothi}@cse.iitb.ac.in Abstract Code-switching has seen growing interest in recent years as an important multilingual NLP phenomenon. Generating code-switched text for data augmentation has been sufficiently well-explored. However, there is no prior work on generating code-switched text with fine-grained control on the degree of code- switching and the lexical choices used to convey formality. We present COCOA, an encoder-decoder translation model that con- verts monolingual Hindi text to Hindi-English code-switched text with both encoder-side and decoder-side interventions to achieve fine- grained controllable generation. COCOA can be invoked at test-time to synthesize code- switched text that is simultaneously faithful to syntactic and lexical attributes relevant to code-switching. COCOA outputs were sub- jected to rigorous subjective and objective evaluations. Human evaluations establish that our outputs are of superior quality while be- ing faithful to desired attributes. We show sig- nificantly improved BLEU scores when com- pared with human-generated code-switched references. Compared to competitive base- lines, we show 10% reduction in perplexity on a language modeling task and also demon- strate clear improvements on a downstream code-switched sentiment analysis task. 1 Introduction Bilingual speakers form a significant portion (cur- rent estimates of 43%1) of the world’s population. To cater to the human-computer interaction needs of this user segment, Natural Language Genera- tion (NLG) and Natural Language Understanding (NLU) tasks for code-switching (CS) are receiving increasing amount of attention from the research community (Zhang et al., 2021). Code-switching in * Equal contribution 1https://ilanguages.org/bilingual.php 0.00 0.25 0.50 Score 0 100 200 300 Count (a) CMI 0.0 0.5 1.0 Score 0 50 100 150 200 Count (b) SPI 0.0 0.5 1.0 Score 0 200 400 600 Count (c) Formality Figure 1: Distribution of CMI, SPI and formality scores over CS samples in the ALLCS test set. a sentence typically involves switching between a matrix language L1 and an embedded language L2. A key challenge in CS that has not been previously addressed by generation models is to explicitly con- trol for syntactic and lexical diversity (Do˘gruöz et al., 2021). Such controllable NLG models would help build robust computational models for CS text. We draw attention to three specific dimensions of diversity that are commonly observed in CS text. In this work, we aim to generate text that specifically spans these three CS dimensions. 1. Language Mix Ratio: The first syntactic di- mension of diversity refers to the varying number of L1 and L2 words in a CS sentence that depends on a number of factors like language pair, socio- economic context, etc. For instance, Al-Azami (2006) find that among immigrant Bengalis in the UK, first-generation immigrants tend to use Ben- gali and English as L1 and L2 respectively, while the order is reversed for younger generation immi- grants. …

Content truncated to 500 words