A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1654–1666 August 1–6, 2021. ©2021 Association for Computational Linguistics 1654 A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies A. Seza Do˘gru¨oz Ghent University, Gent, Belgium as.dogruoz@ugent.be Sunayana Sitaram Microsoft Research India, Bangalore, India sunayana.sitaram@microsoft.com Barbara E. Bullock UT at Austin, Austin, USA bbullock@austin.utexas.edu Almeida Jacqueline Toribio UT at Austin, Austin, USA toribio@austin.utexas.edu Abstract The analysis of data in which multiple lan- guages are represented has gained popularity among computational linguists in recent years. So far, much of this research focuses mainly on the improvement of computational meth- ods and largely ignores linguistic and social aspects of C-S discussed across a wide range of languages within the long-established liter- ature in linguistics. To fill this gap, we offer a survey of code-switching (C-S) covering the literature in linguistics with a reflection on the key issues in language technologies. From the linguistic perspective, we provide an overview of structural and functional patterns of C-S focusing on the literature from European and Indian contexts as highly multilingual areas. From the language technologies perspective, we discuss how massive language models fail to represent diverse C-S types due to lack of appropriate training data, lack of robust evalu- ation benchmarks for C-S (across multilingual situations and types of C-S) and lack of end-to- end systems that cover sociolinguistic aspects of C-S as well. Our survey will be a step to- wards an outcome of mutual benefit for com- putational scientists and linguists with a shared interest in multilingualism and C-S. 1 Introduction It is common for individuals in multilingual com- munities to switch between languages in various ways, in speech and in writing. In example 1, a bilingual child alternates between German and Turkish (in bold) to describe her teacher at school. Note that the Turkish possessive case marker (-si) is attached to a German noun (Karakoc¸ and Herken- rath, 2019). 1. Frau Kummer. Echte Name-si Christa. Ms. Kummer. Real Name-Poss.3sg Christa. ‘Ms. Kummer. (Her) real name is Christa’ The goal of this paper is to inform researchers in computational linguistics (CL) and language tech- nologies about the linguistic and social aspects of code-switching (C-S) found in multilingual con- texts (e.g. Europe and India) and how linguists describe and model them. Our intent is to increase clarity and depth in computational investigations of C-S and to bridge the fields so that they might be mutually reinforcing. It is our hope that interested readers can profit from the insights provided by the studies reported in this survey, for instance, in un- derstanding the factors that guide C-S outcomes or in making use of existing annotation schema across multilingual contexts. 2 Competing models of C-S For linguists, the specific ways in which languages are switched matters. The use of a single Spanish word in an English tweet (ex. 2) is not as syntac- tically complicated as the integration in ex. 1. …

Content truncated to 500 words