Handling verb phrase morphology in highly inflected Indian languages for Machine Translation

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 111–119, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Handling verb phrase morphology in highly inflected Indian languages for Machine Translation Ankur Gandhe, Rashmi Gangadharaiah, Karthik Visweswariah, and Ananthakrishnan Ramanathan IBM Research, India {ankugand,rashgang,v-karthik,aramana2}@in.ibm.com Abstract The phrase based systems for machine translation are limited by the phrases that they see during the training. For highly in- flected languages, it is uncommon to see all the forms of a word in the parallel cor- pora used during training. This problem is amplified for verbs in highly inflected languages where the correct form of the word depends on factors like gender, num- ber and tense aspect. We propose a solu- tion to augment the phrase table with all possible forms of a verb for improving the overall accuracy of the MT system. Our system makes use of simple stemmers and easily available monolingual data to gener- ate new phrase table entries that cover the different variations seen for a verb. We re- port significant gains in BLEU for English to Hindi translation. 1 Introduction Data driven approaches have become widely pop- ular as they use little or no language specific knowledge. The main drawback of these ap- proaches is the need for large amounts of data. (Koehn et al., 2003) have shown that the qual- ity of the translations produced by data driven ap- proaches mainly depends on the amount of parallel data available for the language-pair under consid- eration. Creation of a large bilingual corpus is ex- pensive and time consuming if high quality man- ual translations are required. Hence, building MT systems for language-pairs with limited amounts of data is a big challenge. Approaches have been suggested in the past to mine the world-wide-web to automatically ob- tain large amounts of parallel data. For example, news articles in two different languages describ- ing the same event can be sentence-aligned to ob- tain a parallel corpus. Although this approach has shown improvements, this cannot be extended to languages that have little or no data on the world wide web. The situation gets worse for languages that are rich in morphology. Clearly large amounts of par- allel data are required to observe all variations of a word. Popovic and Ney (2004) applied trans- formations to verbs to reduce the number of out- of-vocabulary words and showed improvements in translation quality when morphemes were consid- ered. Yang and Kirchhoff (2006) used a back off model in a Phrase-based SMT system which trans- lated word forms in the source language by hi- erarchical morphological abstractions. Unknown words in the test data were stemmed and phrase- table entries were modified such that words shar- ing the same root were replaced by their stems. Freeman et al. (2006) and Habash (2008) find in-vocabulary words for OOV words that could be morphological variants of the OOV words. Phrases in the phrase table containing these invo- cabulary words are then replaced by OOV words …

Content truncated to 500 words