A Large Scale Corpus of Gulf Arabic

A Large Scale Corpus of Gulf Arabic Salam Khalifa, Nizar Habash, Dana Abdulrahim†, Sara Hassan Computational Approaches to Modeling Language Lab, New York University Abu Dhabi, UAE †University of Bahrain, Bahrain {salamkhalifa,nizar.habash,sah650}@nyu.edu,darahim@uob.edu.bh Abstract Most Arabic natural language processing tools and resources are developed to serve Modern Standard Arabic (MSA), which is the official written language in the Arab World. Some Dialectal Arabic varieties, notably Egyptian Arabic, have received some attention lately and have a growing collection of resources that include annotated corpora and morphological analyzers and taggers. Gulf Arabic, however, lags behind in that respect. In this paper, we present the Gumar Corpus, a large-scale corpus of Gulf Arabic consisting of 110 million words from 1,200 forum novels. We annotate the corpus for sub-dialect information at the document level. We also present results of a preliminary study in the morphological annotation of Gulf Arabic which includes developing guidelines for a conventional orthography. The text of the corpus is publicly browsable through a web interface we developed for it. Keywords: Arabic Dialects, Corpus, Large-Scale, Gulf Arabic 1. Introduction Most Arabic natural language processing (NLP) tools and resources are developed to serve Modern Standard Arabic (MSA), the official written language in the Arab World. Using such tools to understand and process Dialectal Ara- bic (DA) is a challenging task because of the phonological and morphological differences between DA and MSA. In addition, there is no standard orthography for DA, which only complicates matters more. Some DA varieties, no- tably Egyptian Arabic, have received some attention lately and have a growing collection of resources that include an- notated corpora and morphological analyzers and taggers. Gulf Arabic (GA), broadly defined as the variety of Ara- bic spoken in the countries of the Gulf Cooperation Coun- cil (Bahrain, Kuwait, Oman, Qatar, Saudi Arabia, and the United Arab Emirates), however, lags behind in that re- spect. In this paper, we present the Gumar Corpus,1 a large-scale corpus of GA that includes a number of sub-dialects. We also present preliminary results on GA morphological an- notation. Building a morphologically annotated GA cor- pus is a first step towards developing NLP applications, for searching, retrieving, machine-translating, and spell- checking GA text among other applications. The impor- tance of processing and understanding GA text (as with all DA text) is increasing due to the exponential growth of socially generated dialectal content in social media and printed works (Sarn¯akh, 2014), in addition to existing ma- terials such as folklore and local proverbs that are found scattered on the web. The rest of this paper is structured as follows. We present some related work in Dialectal Arabic NLP in Section 2. This is followed by a background discussion on GA in Sec- tion 3. We then discuss the collection of the corpus and de- scribe its genre in Section 4. We present our preliminary annotation study and evaluate it in Section 5. Finally, we present the Gumar Corpus web interface in Section 6. 1Gumar������ /gumEr/ is the word for ‘moon’ in …

Content truncated to 500 words