Morphology Injection for English-Malayalam Statistical Machine Translation

Morphology Injection for English-Malayalam Statistical Machine Translation Sreelekha S, Pushpak Bhattacharyya IIT Bombay, Mumbai, India {sreelekha, pb} @cse.iitb.ac.in Abstract Statistical Machine Translation (SMT) approaches fails to handle the rich morphology when translating into morphologically rich language. This is due to the data sparsity, which is the missing of the morphologically inflected forms of words from the parallel corpus. We investigated a method to generate these unseen morphological forms. In this paper, we analyze the morphological complexity of a morphologically rich Indian language Malayalam when translating from English. Being a highly agglutinative language, it is very difficult to generate the various morphological inflected forms for Malayalam. We study both the factor based models and the phrase based models and the problem of data sparseness. We propose a simple and effective solution based on enriching the parallel corpus with generated morphological forms. We verify this approach with various experiments on English-Malayalam SMT. We observes that the morphology injection method improves the quality of the translation. We have analyzed the experimental results both in terms of automatic and subjective evaluations. Keywords: Morphology Injection, Statistical Machine Translation, English-Malayalam Machine Translation 1. Introduction Malayalam, Telugu, Kannada and Tamil are being most prominent out of 17 languages in the southern Indian family of Dravidian languages; about 95 per cent of the South Indian population speaks one of these four languages. Out of these over 38 million people is speaking Malayalam primarily in the state of Kerala. Throughout its gradual evolution of the present day Malayalam, the influence of Sanskrit is evident in the alphabet, phonology and vocabulary and to a lesser extent in the morphology of Malayalam. Malayalam is a highly agglutinative and inflectionally rich language with a free word order. The semantic and syntactic relations between the verbs and other constituents in a sentence are represented by the case endings of the words. Vaachakam, which denotes the matter, action and quality, and dyootakam, which denotes the relationships, are the two types of “Sabdam”, a combination of sounds with a meaning. Naamam (noun), kriyaa (verb) and bheedakam (modifier) are the three types of Vaachakam. gati (preposition), ghatakam (conjunction) and vyaakseepakam (interjection) are the three types of dyootakam (Varma, 2000). Malayalam has a strong postpositional inflections with highly agglutinative suffixes (Namboodiri, 1998). These inflections carry information about tense, mood and aspect for verbs and cases (accusative, dative, etc.), gender, number, person information for nouns. Most approaches to Statistical Machine Translation, i.e., phrase based models (Koehn, Och and Marcu, 2003), syntax based models (Yamada and Knight 2001) do not allow incorporation of any linguistic information in the translation process. The introduction of factored models (Koehn and Hoang, 2007) provided this missing linguistic touch to the statistical machine translation. Factored models (Koehn and Hoang, 2007) treat each word in the corpus as vector of tokens. Each token can be any linguistic information about the word which leads to its inflection on the target side. Hence, factored models are preferred over phrase based models (Koehn, Och and Marcu, 2003) while …

Content truncated to 500 words