From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3154–3169 August 1–6, 2021. ©2021 Association for Computational Linguistics 3154 From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text Ishan Tarunesh∗, Syamantak Kumar∗, Preethi Jyothi Samsung Korea, Google India, IIT Bombay {ishantarunesh, syamantak.kumar}@gmail.com, pjyothi@cse.iitb.ac.in Abstract Generating code-switched text is a problem of growing interest, especially given the scarcity of corpora containing large volumes of real code-switched text. In this work, we adapt a state-of-the-art neural machine trans- lation model to generate Hindi-English code- switched sentences starting from monolingual Hindi sentences. We outline a carefully de- signed curriculum of pretraining steps, includ- ing the use of synthetic code-switched text, that enable the model to generate high-quality code- switched text. Using text generated from our model as data augmentation, we show signif- icant reductions in perplexity on a language modeling task, compared to using text from other generative models of CS text. We also show improvements using our text for a down- stream code-switched natural language infer- ence task. Our generated text is further sub- jected to a rigorous evaluation using a human evaluation study and a range of objective met- rics, where we show performance compara- ble (and sometimes even superior) to code- switched text obtained via crowd workers who are native Hindi speakers. 1 Introduction Code-switching (CS) refers to the linguistic phe- nomenon of using more than one language within a single sentence or conversation. CS appears natu- rally in conversational speech among multilingual speakers. The main challenge with building mod- els for conversational CS text is that we do not have access to large amounts of CS text that is con- versational in style. One might consider using so- cial media text that contains CS and is more read- ily available. However, the latter is quite different from conversational CS text in its vocabulary (e.g., due to the frequent use of abbreviated slang terms, ∗Work done while first two authors were students at IIT Bombay. hashtags and mentions), in its sentence structure (e.g., due to character limits in tweets) and in its word forms (e.g., due to transliteration being com- monly employed in social media posts). This mo- tivates the need for a generative model of realistic CS text that can be sampled to subsequently train models for CS text. In this work, we tackle the problem of gen- erating high-quality CS text using only limited amounts of real CS text during training. We also assume access to large amounts of monolingual text in the component languages and parallel text in both languages, which is a reasonable assump- tion to make for many of the world’s languages. We focus on Hindi-English CS text where the ma- trix (dominant) language is Hindi and the embed- ded language is English.1 Rather than train a gen- erative model, we treat this problem as a transla- tion task where the source and target languages are monolingual Hindi text and Hindi-English …

Content truncated to 500 words