From English to Code-Switching: Transfer Learning with Strong Morphological Clues

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8033–8044 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8033 From English to Code-Switching: Transfer Learning with Strong Morphological Clues Gustavo Aguilar and Thamar Solorio Department of Computer Science University of Houston Houston, TX 77204-3010 {gaguilaralas, tsolorio}@uh.edu Abstract Linguistic Code-switching (CS) is still an un- derstudied phenomenon in natural language processing. The NLP community has mostly focused on monolingual and multi-lingual sce- narios, but little attention has been given to CS in particular. This is partly because of the lack of resources and annotated data, despite its in- creasing occurrence in social media platforms. In this paper, we aim at adapting monolin- gual models to code-switched text in various tasks. Specifically, we transfer English knowl- edge from a pre-trained ELMo model to differ- ent code-switched language pairs (i.e., Nepali- English, Spanish-English, and Hindi-English) using the task of language identification. Our method, CS-ELMo, is an extension of ELMo with a simple yet effective position-aware at- tention mechanism inside its character convo- lutions. We show the effectiveness of this transfer learning step by outperforming mul- tilingual BERT and homologous CS-unaware ELMo models and establishing a new state of the art in CS tasks, such as NER and POS tag- ging. Our technique can be expanded to more English-paired code-switched languages, pro- viding more resources to the CS community. 1 Introduction Although linguistic code-switching (CS) is a com- mon phenomenon among multilingual speakers, it is still considered an understudied area in natural language processing. The lack of annotated data combined with the high diversity of languages in which this phenomenon can occur makes it diffi- cult to strive for progress in CS-related tasks. Even though CS is largely captured in social media plat- forms, it is still expensive to annotate a sufficient amount of data for many tasks and languages. Ad- ditionally, not all the languages have the same in- cidence and predominance, making annotations impractical and expensive for every combination Hindi-English Tweet Original: Keep calm and keep kaam se kaam !!!other #office #tgif #nametag #buddhane #SouvenirFromManali #keepcalm English: Keep calm and mind your own business !!! Nepali-English Tweet Original: Youtubene ma live re ,other chalcha ki vanni aash garam !other Optimistic .other English: They said Youtube live, let’s hope it works! Optimistic. Spanish-English Tweet Original: @MROlvera06other @T11gReother go too cavendersne y tambien ve a @ElToroBootsne other English: @MROlvera06 @T11gRe go to cavenders and also go to @ElToroBoots Figure 1: Examples of code-switched tweets and their translations from the CS LID corpora for Hindi- English, Nepali-English and Spanish-English. The LID labels ne and other in subscripts refer to named enti- ties and punctuation, emojis or usernames, respectively (they are part of the LID tagset). English text appears in italics and other languages are underlined. of languages. Nevertheless, code-switching often occurs in language pairs that include English (see examples in Figure 1). These aspects lead us to ex- plore approaches where English pre-trained models can be leveraged and tailored …

Content truncated to 500 words