An Arabic-Moroccan Darija Code-Switched Corpus

An Arabic-Moroccan Darija Code-Switched Corpus Younes Samih and Wolfgang Maier Institute for Language and Information University of D¨usseldorf, D¨usseldorf, Germany {samih,maierwo}@phil.hhu.de Abstract In multilingual communities, speakers often switch between languages or dialects within the same context. This phenomenon is called code-switching. It can be observed, e.g., in the Arab world, where Modern Standard Arabic and Dialectal Arabic coexist. Recently, the computational treatment of code-switching has received attention. Just as other natural language processing tasks, this task requires annotated linguistic resources. In our work, we turn to a particular under-resourced Arabic Dialect, Moroccan Darija. While other dialects such as Egyptian Arabic have received their share of attention, very limited effort has been devoted to the development of basic linguistic resources that would support a computational treatment of Darija. Motivated by these considerations, we describe our effort in the development and annotation of a large scale corpus collected from Moroccan social media sources, namely blogs and internet discussion forums. It has been annotated on token-level by three Darija native speakers. Crowd-sourcing has not been used. The final corpus has a size of 223k tokens. It is, to our knowledge, currently the largest resource of its kind. Keywords: code-switching, language identification, Moroccan Arabic 1. Introduction Modern Standard Arabic (MSA) is the official language of most Arabic countries. It is spoken by more than 360 mil- lion people around the world and exists in state of diglossia (Ferguson, 1959). Arabic speakers tend to use Dialectal Arabic (DA) and MSA, two substantially different but his- torically related language varieties, for different purposes as the situation demands in their day-to-day lives. Moroccan Arabic Gulf Arabic Iraqi Arabic Egyptian Arabic Yemeni Arabic Other Other Other uT n aise n Algerian Arabic Lybian Arabic Levantine Arabic Figure 1: The Arab world and Arabic Dialects Fig. 1 shows a schematic map of dialects. Note that of- ten Moroccan, Algerian, Tunisian and Lybian Arabic are grouped together as Maghrebi Arabic, even though they are not necessarily mutually intelligible. While MSA is an es- tablished standard among educated Arabic speakers, DA is only used in everyday informal communication. Until re- cently, DA was considered as a partially under-resourced language, as the written production remains relatively very low in comparison to MSA. Increasingly, however, DA is emerging as the language of informal communication on the web, in emails, micro-blogs, blogs, forums, chat rooms, etc. This new situation amplifies the need for con- sistent language resources and language identification sys- tems for Arabic and its dialects. While certain dialects, par- ticularly Egyptian, have already received attention in NLP research, Moroccan Arabic (Darija) (Ennaji et al., 2004; Benmamoun, 2001), a dialect with over 21 million native speakers (Lewis et al., 2014), remains a particularly under- resourced variant of Arabic. It is strongly embedded in a multilingual context that entails frequent code-switching, i.e., switching between languages within the same context (Bullock and Toribio, 2009). Building linguistic resources and creating the necessary tools for Darija is a priority, not least because its vocabulary is particularly distant …

Content truncated to 500 words