Morphological Analysis and Disambiguation for Dialectal Arabic

Proceedings of NAACL-HLT 2013, pages 426–432, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Morphological Analysis and Disambiguation for Dialectal Arabic Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh Center for Computational Learning Systems Columbia University {habash,ryanr,rambow,reskander,nadi}@ccls.columbia.edu Abstract The many differences between Dialectal Ara- bic and Modern Standard Arabic (MSA) pose a challenge to the majority of Arabic natural language processing tools, which are designed for MSA. In this paper, we retarget an exist- ing state-of-the-art MSA morphological tag- ger to Egyptian Arabic (ARZ). Our evalua- tion demonstrates that our ARZ morphology tagger outperforms its MSA variant on ARZ input in terms of accuracy in part-of-speech tagging, diacritization, lemmatization and to- kenization; and in terms of utility for ARZ-to- English statistical machine translation. 1 Introduction Dialectal Arabic (DA) refers to the day-to-day na- tive vernaculars spoken in the Arab World. DA is used side by side with Modern Standard Arabic (MSA), the official language of the media and edu- cation (Holes, 2004). Although DAs are historically related to MSA, there are many phonological, mor- phological and lexical differences between them. Unlike MSA, DAs have no standard orthographies or language academies. Furthermore, different DAs, such as Egyptian Arabic (henceforth, ARZ), Levan- tine Arabic or Moroccan Arabic have important dif- ferences among them, similar to those seen among Romance languages (Holes, 2004; Abdel-Massih et al., 1979). Most tools and resources developed for natural language processing (NLP) of Arabic are de- signed for MSA. Such resources are quite limited when it comes to processing DA, e.g., a state-of- the-art MSA morphological analyzer only has 60% coverage of Levantine Arabic verb forms (Habash and Rambow, 2006). In this paper, we describe the process of retar- geting an existing state-of-the-art tool for model- ing MSA morphology disambiguation to ARZ, the most commonly spoken DA. The MSA tool we extend is MADA – Morphological Analysis and Disambiguation of Arabic (Habash and Rambow, 2005). The approach used in MADA, which was inspired by earlier work by Hajiˇc (2000), disam- biguates in context for every aspect of Arabic mor- phology, thus solving all tasks in “one fell swoop”. The disadvantage of the MADA approach is its de- pendence on two complex resources: a morpholog- ical analyzer for the language and a large collection of manually annotated words for all morphological features in the same representation used by the an- alyzer. For ARZ, such resources have recently be- come available, with the development of the CAL- IMA ARZ morphological analyzer (Habash et al., 2012b) and the release by the Linguistic Data Con- sortium (LDC) of a large ARZ corpus annotated morphologically in a manner compatible with CAL- IMA (Maamouri et al., 2012a). In the work pre- sented here, we utilize these new resources within the paradigm of MADA, transforming MADA into MADA-ARZ. The elegance of the MADA solution makes this conceptually a simple extension. Our evaluation demonstrates that our Egyptian DA version of MADA, henceforth MADA-ARZ, outperforms MADA for MSA on ARZ morpholog- ical …

Content truncated to 500 words