A Hybrid Morphologically Decomposed Factored Language Models for Arabic LVCSR

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 701–704, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics A Hybrid Morphologically Decomposed Factored Language Models for Arabic LVCSR Amr El-Desoky, Ralf Schl¨uter, Hermann Ney Lehrstuhl f¨ur Informatik 6 – Computer Science Department RWTH Aachen University, D-52056 Aachen, Germany {desoky,schluter,ney}@cs.rwth-aachen.de Abstract In this work, we try a hybrid methodology for language modeling where both morphological decomposition and factored language model- ing (FLM) are exploited to deal with the com- plex morphology of Arabic language. At the end, we are able to obtain from 3.5% to 7.0% relative reduction in word error rate (WER) with respect to a traditional full-words sys- tem, and from 1.0% to 2.0% relative WER re- duction with respect to a non-factored decom- posed system. 1 Introduction Arabic language is characterized by a complex mor- phological structure where different kinds of pre- fixes and suffixes are appended to the word stems producing a very large number of inflectional forms. This leads to poor language model (LM) probabil- ity estimates, and thus high LM perplexities (PPLs) causing problems in large vocabulary continuous speech recognition (LVCSR). One successful ap- proach to deal with this problem is to consider LMs including morphologically decomposed words. An- other approach is to use the factored language mod- els (FLMs) which are powerful models that com- bine multiple sources of information and efficiently integrate them via a complex backoff mechanism (Bilmes and Kirchhoff, 2003). Morphological decomposition is successfully used for Arabic LMs in several previous works. Some are based on linguistic knowledge, and oth- ers are based on unsupervised methods. Some of the linguistic methods are based on the Buckwalter Ara- bic Morphological Analyzer (BAMA) like in (Lamel et al., 2008). Alternatively, in our previous work (El-Desoky et al., 2009), we use the Morphological Analyzer and Disambiguator for Arabic (MADA) (Habash and Rambow, 2007). On the other side, most of the unsupervised methods are based on the minimum description length principle (MDL) like in (Creutz et al., 2007). Another type of models is the FLM, in which words are viewed as vectors of K factors, so that wt := {f1:K t }. A factor could be any feature of the word such as morphological class, stem, root or even a semantic feature. An FLM is a model over factors, i.e., p(f1:K t |f1:K t−1 , f1:K t−2 , ..., f1:K t−n+1), which could be reformed as a product of probabilities of the form p(f|f1, f2, ..., fN). The main idea of the model is to backoff to other factors when some word n-gram is not observed in the training data, thus improving the probability estimates. In this work we try to combine the strengths of morphological decomposition and factored lan- guage modeling. Therefore, language models with factored morphemes are used. For this purpose, the LM training data are processed such that full-words are decomposed into prefix-stem-suffix format with different added features. We compare our approach with …

Content truncated to 500 words