Machine Translation of Arabic Dialects

2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 49–59, Montr´eal, Canada, June 3-8, 2012. c⃝2012 Association for Computational Linguistics Machine Translation of Arabic Dialects Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F. Zaidan†, Chris Callison-Burch‡ Raytheon BBN Technologies, Cambridge MA †Microsoft Research, Redmond WA ‡Johns Hopkins University, Baltimore MD Abstract Arabic Dialects present many challenges for machine translation, not least of which is the lack of data resources. We use crowdsourc- ing to cheaply and quickly build Levantine- English and Egyptian-English parallel cor- pora, consisting of 1.1M words and 380k words, respectively. The dialectal sentences are selected from a large corpus of Arabic web text, and translated using Amazon’s Mechan- ical Turk. We use this data to build Dialec- tal Arabic MT systems, and find that small amounts of dialectal data have a dramatic im- pact on translation quality. When translating Egyptian and Levantine test sets, our Dialec- tal Arabic MT system performs 6.3 and 7.0 BLEU points higher than a Modern Standard Arabic MT system trained on a 150M-word Arabic-English parallel corpus. 1 Introduction The Arabic language is a well-known example of diglossia (Ferguson, 1959), where the formal vari- ety of the language, which is taught in schools and used in written communication and formal speech (religion, politics, etc.) differs significantly in its grammatical properties from the informal varieties that are acquired natively, which are used mostly for verbal communication. The spoken varieties of the Arabic language (which we refer to collectively as Dialectal Arabic) differ widely among themselves, depending on the geographic distribution and the socio-economic conditions of the speakers, and they diverge from the formal variety known as Mod- ern Standard Arabic (MSA) (Embarki and Ennaji, 2011). Significant differences in the phonology, morphology, lexicon and even syntax render some of these varieties mutually incomprehensible. The use of Dialectal Arabic has traditionally been confined to informal personal speech, while writ- ing has been done almost exclusively using MSA (or its ancestor Classical Arabic). This situation is quickly changing, however, with the rapid prolifer- ation of social media in the Arabic-speaking part of the world, where much of the communication is composed in dialect. The focus of the Arabic NLP research community, which has been mostly on MSA, is turning towards dealing with informal com- munication, with the introduction of the DARPA BOLT program. This new focus presents new chal- lenges, the most obvious of which is the lack of di- alectal linguistic resources. Dialectal text, which is usually user-generated, is also noisy, and the lack of standardized orthography means that users often improvise spelling. Dialectal data also includes a wider range of topics than formal data genres, such as newswire, due to its informal nature. These chal- lenges require innovative solutions if NLP applica- tions are to deal with Dialectal Arabic effectively. In this paper: • We describe a process for cheaply and quickly developing parallel corpora for Levantine- English and …

Content truncated to 500 words