Morpho-syntactic Arabic Preprocessing for Arabic to English Statistical Machine Translation

Proceedings of the Workshop on Statistical Machine Translation, pages 15–22, New York City, June 2006. c⃝2006 Association for Computational Linguistics Morpho-syntactic Arabic Preprocessing for Arabic-to-English Statistical Machine Translation Anas El Isbihani Shahram Khadivi Oliver Bender Lehrstuhl f¨ur Informatik VI - Computer Science Department RWTH Aachen University D-52056 Aachen, Germany {isbihani,khadivi,bender,ney}@informatik.rwth-aachen.de Hermann Ney Abstract The Arabic language has far richer sys- tems of inflection and derivation than En- glish which has very little morphology. This morphology difference causes a large gap between the vocabulary sizes in any given parallel training corpus. Segmen- tation of inflected Arabic words is a way to smooth its highly morphological na- ture. In this paper, we describe some statistically and linguistically motivated methods for Arabic word segmentation. Then, we show the efficiency of proposed methods on the Arabic-English BTEC and NIST tasks. 1 Introduction Arabic is a highly inflected language compared to English which has very little morphology. This mor- phological richness makes statistical machine trans- lation from Arabic to English a challenging task. A usual phenomenon in Arabic is the attachment of a group of words which are semantically dependent on each other. For instance, prepositions like “and” and “then” are usually attached to the next word. This applies also to the definite article “the”. In addi- tion, personal pronouns are attached to the end of verbs, whereas possessive pronouns are attached to the end of the previous word, which constitutes the possessed object. Hence, an Arabic word can be de- composed into “prefixes, stem and suffixes”. We re- strict the set of prefixes and suffixes to those showed in Table 1 and 2, where each of the prefixes and suf- fixes has at least one meaning which can be repre- sented by a single word in the target language. Some prefixes can be combined. For example the word wbAlqlm (��������� which means “and with the pen”) has a prefix which is a combination of three pre- fixes, namely w, b and Al. The suffixes we handle in this paper can not be combined with each other. Thus, the compound word pattern handled here is “prefixes-stem-suffix“. All possible prefix combinations that do not con- tain Al allow the stem to have a suffix. Note that there are other suffixes that are not handled here, such as At ( ���), An ( ���) and wn ( ���) which make the plural form of a word. The reason why we omit them is that they do not have their own meaning. The impact of Arabic morphology is that the vocabulary size and the number of singletons can be dramati- cally high, i.e. the Arabic words are not seen often enough to be learned by statistical machine transla- tion models. This can lead to an inefficient align- ment. In order to deal with this problem and to improve the performance of statistical machine translation, each word must be decomposed into its parts. In (Larkey et al., 2002) it was already shown that word segmentation for Arabic improves information …

Content truncated to 500 words