The MADAR Arabic Dialect Corpus and Lexicon

The MADAR Arabic Dialect Corpus and Lexicon Houda Bouamor, Nizar Habash,† Mohammad Salameh, Wajdi Zaghouani,♣ Owen Rambow,‡ Dana Abdulrahim,∗ Ossama Obeid,† Salam Khalifa,† Fadhl Eryani,† Alexander Erdmann,† Kemal Oflazer Carnegie Mellon University in Qatar, Qatar, ♣Hamad Bin Khalifa University, Qatar †New York University Abu Dhabi, UAE ‡Columbia University, USA, ∗University of Bahrain, Bahrain hbouamor@cmu.edu, nizar.habash@nyu.edu, wzaghouano@hbku.edu.qa, rambow@cs.columbia.edu, darahim@uob.edu.bh Abstract In this paper, we present two resources that were created as part of the Multi Arabic Dialect Applications and Resources (MADAR) project. The first is a large parallel corpus of 25 Arabic city dialects in the travel domain. The second is a lexicon of 1,045 concepts with an average of 45 words from 25 cities per concept. These resources are the first of their kind in terms of the breadth of their coverage and the fine location granularity. The focus on cities, as opposed to regions in studying Arabic dialects, opens new avenues to many areas of research from dialectology to dialect identification and machine translation. Keywords: Arabic Dialects, Parallel Corpus, Lexicon 1. Introduction Dialectal Arabic (DA) is emerging nowadays as the primary written language of informal communication online in the Arab World: in emails, blogs, discussion forums, chats, SMS, etc. There has been a rising interest in research on computational models of Arabic dialects in the last decade (Meftouh et al., 2015). There have been several efforts on creating different resources to allow building models for several Natural Language Processing (NLP) applications. However, these efforts have been disjoint from each other, and most of them have focused on a small number of di- alects that represent vast regions of the Arab World (Zaidan and Callison-Burch, 2011; Diab et al., 2014; Sajjad et al., 2013). In this paper, we present two resources we created as part of the Multi Arabic Dialect Applications and Resources (MADAR) project.1 The goal of MADAR is to create, for a large number of dialects, a unified framework with common annotation guidelines and decisions, and targeting applica- tions of Dialect Identification (DID) and Machine Transla- tion (MT). The first resource is a large parallel corpus of 25 Arabic city dialects, in addition to the pre-existing parallel set for English, French and Modern Standard Arabic (MSA). The second resource is a 25-way lexicon of 1,045 entries in each city’s dialect along with MSA, French, and English. These resources are the first of their kind in terms of the breadth of their coverage and their fine granularity. The kind of resources we present in this paper are useful not only for building computational systems but also for studying Ara- bic dialects from a linguistics perspective (e.g., computa- tional dialectology). 1The project site is at http://nlp.qatar.cmu.edu/madar/. �� ���� madAr means ‘orbit’ in Arabic. 2. Arabic and its Dialects The Arabic language is a family of varieties. Among these varieties, Modern Standard Arabic (MSA) is the shared language of culture, media, and education from Morocco to Oman. However, MSA is not the native language of any speaker of Arabic. In unscripted …

Content truncated to 500 words