Dialectal Arabic to English Machine Translation: Pivoting through Modern Standard Arabic

Proceedings of NAACL-HLT 2013, pages 348–358, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Dialectal Arabic to English Machine Translation: Pivoting through Modern Standard Arabic Wael Salloum and Nizar Habash Center for Computational Learning Systems Columbia University {wael,habash}@ccls.columbia.edu Abstract Modern Standard Arabic (MSA) has a wealth of natural language processing (NLP) tools and resources. In comparison, resources for dialectal Arabic (DA), the unstandardized spo- ken varieties of Arabic, are still lacking. We present ELISSA, a machine translation (MT) system for DA to MSA. ELISSA employs a rule-based approach that relies on morpho- logical analysis, transfer rules and dictionar- ies in addition to language models to produce MSA paraphrases of DA sentences. ELISSA can be employed as a general preprocessor for DA when using MSA NLP tools. A man- ual error analysis of ELISSA’s output shows that it produces correct MSA translations over 93% of the time. Using ELISSA to produce MSA versions of DA sentences as part of an MSA-pivoting DA-to-English MT solution, improves BLEU scores on multiple blind test sets between 0.6% and 1.4%. 1 Introduction Much work has been done on Modern Standard Ara- bic (MSA) natural language processing (NLP) and machine translation (MT), especially Statistical MT (SMT). MSA has a wealth of resources in terms of morphological analyzers, disambiguation systems, and parallel corpora. In comparison, research on di- alectal Arabic (DA), the unstandardized spoken vari- eties of Arabic, is still lacking in NLP in general and MT in particular. In this paper we present ELISSA, our DA-to-MSA MT system, and show how it can help improve the translation of highly dialectal Ara- bic text into English by pivoting on MSA. The ELISSA approach can be summarized as fol- lows. First, ELISSA uses different techniques to identify dialectal words and multi-word construc- tions (phrases) in a source sentence. Then, ELISSA produces MSA paraphrases for the selected words and phrase using a rule-based component that de- pends on the existence of a dialectal morphologi- cal analyzer, a list of morphosyntactic transfer rules, and DA-MSA dictionaries. The resulting MSA is in a lattice form that we pass to a language model for n- best decoding. The output of ELISSA, whether a top- 1 choice sentence or n-best sentences, is passed to an MSA-English SMT system to produce the English translation sentence. ELISSA-based MSA-pivoting for DA-to-English SMT improves BLEU scores (Pa- pineni et al., 2002) on three blind test sets between 0.6% and 1.4%. A manual error analysis of trans- lated words shows that ELISSA produces correct MSA translations over 93% of the time. The rest of this paper is structured as follows: Section 2 motivates the use of ELISSA to improve DA-English SMT with an example. Section 3 dis- cusses some of the challenges associated with pro- cessing Arabic and its dialects. Section 4 presents related work. Section 5 details ELISSA and its approach and Section 6 presents results evaluating ELISSA under a variety of conditions. 2 Motivating Example Table 1 shows a motivating example of how pivot- …

Content truncated to 500 words