Collecting Code-Switched Data from Social Media

Collecting Code-Switched Data from Social Media Gideon Mendels†, Victor Soto†, Aaron Jaech, Julia Hirschberg Columbia University, New York, NY 10027 University of Washington, Seattle, WA 98105 g.mendels@columbia.edu, vsoto@cs.columbia.edu, ajaech@uw.edu, julia@cs.columbia.edu Abstract We address the problem of mining code-switched data from the web, where code-switching is defined as the tendency of bilinguals to switch between their multiple languages both across and within utterances. We propose a method that identifies data as code-switched in languages L1 and L2 when a language classifier labels the document as language L1 but the document also contains words that can only belong to L2. We apply our method to Twitter data and collect a set of more than 43,000 tweets. We obtain language identifiers for a subset of 8,000 tweets using crowd-sourcing with high inter-annotator agreement and accuracy. We validate our Twitter corpus by comparing it to the Spanish-English corpus of code-switched tweets collected for the EMNLP 2016 Shared Task for Language Identification, in terms of code-switching rates, language composition and amount of code-switch types found in both datasets. We then trained language taggers on both corpora and show that a tagger trained on the EMNLP corpus exhibits a considerable drop in accuracy when tested on the new corpus and a tagger trained on our new corpus achieves very high accuracy when tested on both corpora. Keywords: code-switching, data collection, social media 1. Introduction Linguistic code-switching is the phenomenon by which bilingual speakers switch back and forth between languages during communication. Code-switching can be classified as inter-sentential when the switch occurs between the bound- aries of a sentence or utterance, or intra-sentential when it occurs within those boundaries. For example a Spanish- English speaker might say “Me dijo que Juanito is very good at math,” which represents an intra-sentential switch, or “Me dijo que Juanito minti´o. I don’t believe it!” Code-switching can be observed at various linguistic lev- els of representation for different language pairs: phono- logical, morphological, lexical, syntactic, semantic, and discourse/pragmatic switching. However, very little code- switching corpora exist from which researchers can train statistical models. The question of how to acquire code- switched data from web and social media resources auto- matically and accurately remains largely unaddressed. In this paper, we present a method to automatically col- lect code-switched data from Twitter. Twitter data has been mined extensively for many Natural Language Processing and speech tasks (Mendels et al., 2015; Go et al., 2009) as one of the only major platforms that provides an API for data collection. The proposed method, which we term “anchoring” can also be used for collecting data from other sources. The remainder of the paper is organized as follows. In Sec- tion 2. we give an overview of previous work on the topic of finding and collecting code-switched data. In Section 3. we present our anchoring method for retrieving code- switched tweets. Section 4. provides the details of our Twitter collection pipeline. Section 5. describes the lan- guage identification (LID) task we used to crowdsource the word language …

Content truncated to 500 words