Machine Translation Experiments on PADIC: A Parallel Arabic DIalect Corpus

Machine Translation Experiments on PADIC: A Parallel Arabic DIalect Corpus Karima Meftouh Badji Mokhtar University Annaba, Algeria Salima Harrat Ecole Sup´erieure d’Informatique (ESI), ENSB∗ Algiers, Algeria Salma Jamoussi MIRACL† Pole Technologique de Sfax, Tunisia Mourad Abbas CRSTDLA‡ Algiers, Algeria Kamel Smaili Campus Scientifique LORIA Nancy, France Abstract We present in this paper PADIC, a Parallel Arabic DIalect Corpus we built from scratch, then we conducted experiments on cross- dialect Arabic machine translation. PADIC is composed of dialects from both the Maghreb and the Middle-East. Each dialect has been aligned with Modern Standard Arabic (MSA). Three dialects from Maghreb are concerned by this study: two from Algeria, one from Tunisia, and two dialects from the Middle- East (Syria and Palestine). PADIC has been built from scratch because the lack of dialect resources. In fact, Arabic dialects in Arab world in general are used in daily life con- versations but they are not written. At the best of our knowledge, PADIC, up to now, is the largest corpus in the community work- ing on dialects and especially those concern- ing Maghreb. PADIC is composed of 6400 sentences for each of the 5 concerned dialects and MSA. We conducted cross-lingual ma- chine translation experiments between all the language pairs. For translating to MSA we in- terpolated the corresponding Language Model (LM) with a large Arabic corpus based LM. We also studied the impact of language model smoothing techniques on the results of ma- chine translation because this corpus, even it is the largest one, it still very small in com- parison to those used for translation of natural languages. ∗Ecole Normale Sup´erieure Bouzareah. †Multimedia, InfoRmation systems and Advanced Comput- ing Laboratory. ‡Centre de Recherche Scientifique et Technique pour le D´eveloppement de la Langue Arabe. 1 Introduction Recently, in addition of translating MSA (Modern Standard Arabic), a new challenging issue emerges: How to deal with the translation of Arabic dialects? Few years ago, some works have been proposed to process Arabic dialects and more specifically those of Middle-East. These works concerned building lexicon, analyzing text morphology, POS tagging, diacritization, etc, (Kilany et al., 2002; Kirchhoff et al., 2003; Habash and Rambow, 2006; Chiang et al., 2006; Habash et al., 2013; Elfardy and Diab, 2013; Pasha et al., 2014; Harrat et al., 2014). In the context of Machine Translation some Arabic dialects have started receiving increasing attention (Sawaf, 2010; Zbib et al., 2012; Salloum and Habash, 2013). Number of researchers have exploited the NLP tools dedicated to MSA to develop their Machine Trans- lation (MT) systems for Arabic dialects, considered as under-resourced languages. Ridouane and Karim (2014) used tools designed for MSA and adapted them to Moroccan dialect in order to build a trans- lation system from MSA to Moroccan, by combin- ing a rule-based approach and a statistical approach. Sawaf (2010) built a hybrid AD-English MT system that uses a MSA pivot approach. In this approach, AD is transferred into MSA using character-based AD normalization rules. In addition an AD normal- ization decoder that …

Content truncated to 500 words