Parsing Arabic Dialects

Parsing Arabic Dialects David Chiang∗, Mona Diab†, Nizar Habash†, Owen Rambow†, Safiullah Shareef‡ ∗ ISI, University of Southern California † CCLS, Columbia University ‡ The Johns Hopkins University chiang@isi.edu, {mdiab,habash,rambow}@cs.columbia.edu, safi@jhu.edu Abstract The Arabic language is a collection of spoken dialects with important phonolog- ical, morphological, lexical, and syntac- tic differences, along with a standard writ- ten language, Modern Standard Arabic (MSA). Since the spoken dialects are not officially written, it is very costly to obtain adequate corpora to use for training dialect NLP tools such as parsers. In this paper, we address the problem of parsing tran- scribed spoken Levantine Arabic (LA). We do not assume the existence of any anno- tated LA corpus (except for development and testing), nor of a parallel corpus LA- MSA. Instead, we use explicit knowledge about the relation between LA and MSA. 1 Introduction: Arabic Dialects The Arabic language is a collection of spoken dialects and a standard written language.1 The dialects show phonological, morphological, lexi- cal, and syntactic differences comparable to those among the Romance languages. The standard written language is the same throughout the Arab world: Modern Standard Arabic (MSA). MSA is also used in some scripted spoken communica- tion (news casts, parliamentary debates). MSA is based on Classical Arabic and is not a native lan- guage of any Arabic speaking people, i.e., children do not learn it from their parents but in school. 1This paper is based on work done at the 2005 Johns Hop- kins Summer Workshop, which was partially supported by the National Science Foundation under Grant No. 0121285. Diab, Habash, and Rambow were supported for additional work by DARPA contract HR0011-06-C-0023 under the GALE program. We wish to thank audiences at JHU for their useful feedback. The authors are listed in alphabetical order. Most native speakers of Arabic are unable to pro- duce sustained spontaneous MSA. Dialects vary not only along a geographical continuum but also with other sociolinguistic variables such as the ur- ban/rural/Bedouin dimension. The multidialectal situation has important neg- ative consequences for Arabic natural language processing (NLP): since the spoken dialects are not officially written and do not have standard or- thography, it is very costly to obtain adequate cor- pora, even unannotated corpora, to use for train- ing NLP tools such as parsers. Furthermore, there are almost no parallel corpora involving one di- alect and MSA. In this paper, we address the problem of parsing transcribed spoken Levantine Arabic (LA), which we use as a representative example of the Arabic dialects.2 Our work is based on the assumption that it is easier to manually create new resources that relate LA to MSA than it is to manually cre- ate syntactically annotated corpora in LA. Our ap- proaches do not assume the existence of any anno- tated LA corpus (except for development and test- ing), nor of a parallel LA-MSA corpus. Instead, we assume we have at our disposal a lexicon that relates LA lexemes to MSA lexemes, and knowl- edge about the morphological …

Content truncated to 500 words