Morphologically Annotated Corpora and Morphological Analyzers for Moroccan and Sanaani Yemeni Arabic

Morphologically Annotated Corpora and Morphological Analyzers for Moroccan and Sanaani Yemeni Arabic Faisal Al-Shargi∗, Aidan Kaplan†, Ramy Eskander‡, Nizar Habash§, Owen Rambow‡ ∗Universität Leipzig, Germany; †Yale University, USA; ‡Columbia University, USA § New York University Abu Dhabi, United Arab Emirates alshargi@informatik.uni-leipzig.de, aidan.kaplan@yale.edu, rnd2110@columbia.edu nizar.habash@nyu.edu, rambow@ccls.columbia.edu Abstract We present new language resources for Moroccan and Sanaani Yemeni Arabic. The resources include corpora for each dialect which have been morphologically annotated, and morphological analyzers for each dialect which are derived from these corpora. These are the first sets of resources for Moroccan and Yemeni Arabic. The resources will be made available to the public. Keywords: Corpus, Arabic Dialects, Morphological Analysis 1. Introduction Traditionally, Arabic dialects have been mainly spoken, and only rarely written. Most written Arabic has been Classi- cal Arabic or Modern Standard Arabic (MSA), and there- fore most natural language processing (NLP) tools address these two variants of Arabic. However, this situation is currently undergoing change. The expanding use of social media and electronic forms of written communication has been accompanied in the Arab world by an increase in the production of written dialect. As a result, there is a grow- ing interest in NLP tools for written Arabic dialects, for example from companies that would like to perform sen- timent data mining on regional web forums to determine what consumers think about their products. However, few such NLP tools for Arabic dialects exist. An important fac- tor that contributes to the lack of NLP tools for the dialects is that there are many distinct dialects and few annotated corpora. This paper presents coordinated resource creation efforts for two Arabic dialects: Moroccan (MOR) and the Sanaani Dialect of Yemen (YEMS). For each dialect, we present a new morphologically annotated corpus and a mor- phological analyzer. These are the first such resources for Moroccan and for Yemeni, to our knowledge. This paper is structured as follows. We start with related work (Section 2.). We then present some linguistic facts for our two dialects (Section 3.). Section 4. contains de- tails about our two corpora, and we sketch our annotation scheme in Section 5.. We discuss the creation of the mor- phological analyzers in Section 6., and then conclude. The conclusion also contains information about how to obtain our resources. 2. Related Work There has been a fairly large amount of descriptive work on Moroccan Arabic, with prominent publications includ- ing a reference grammar (Harrell, 1962) and a dictionary (Harrell et al., 2004), as well as in-depth work on Moroc- can syntax (Brustad, 2000). For Yemeni Arabic, there has also been a fair amount of work in theoretical and descrip- tive linguistics (Jastrow, 1984; Behnstedt, 1985; Abdullah, 1991; Watson, 1993; Naïm-Sanbar, 1994; Al-Iryani, 1996; Behnstedt, 2006). There have been several data collections centered on Arabic dialects, specifically spoken Arabic. A very useful resource is the Semitisches Tonarchiv at the University of Heidel- berg in Germany1 under the direction of Prof. Werner Arnold. We have included one Yemeni transcription from this resource in …

Content truncated to 500 words