Handling Indonesian Clitics: A Dataset Comparison for an Indonesian-English Statistical Machine Translation System

Copyright 2012 by Septina Dian Larasati 26th Pacific Asia Conference on Language,Information and Computation pages 146–152 Handling Indonesian Clitics: A Dataset Comparison for an Indonesian-English Statistical Machine Translation System Septina Dian Larasati Charles University in Prague, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics Prague, Czech Republic SIA TILDE Riga, Latvia larasati@ufal.mff.cuni.cz, septina@tilde.lv Abstract In this paper, we study the effect of incorporat- ing morphological information on an Indone- sian (id) to English (en) Statistical Machine Translation (SMT) system as part of a prepro- cessing module. The linguistic phenomenon that is being addressed here is Indonesian cliti- cized words. The approach is to transform the text by separating the correct clitics from a cliticized word to simplify the word align- ment. We also study the effect of applying the preprocessing on different SMT systems trained on different kinds of text, such as spo- ken language text. The system is built using the state-of-the-art SMT tool, MOSES. The Indonesian morphological information is pro- vided by MorphInd. Overall the preprocessing improves the translation quality, especially for the Indonesian spoken language text, where it gains 1.78 BLEU score points of increase. 1 Introduction Incorporating linguistic information into statistical Natural Language Processing (NLP) applications usually helps to improve a particular NLP. Simpli- fying the problem beforehand, for languages with complex language constructions, is one of the ap- proaches that is usually applied, especially when the constructions cannot be represented by a statistical model. Incorporating morphological information as part of a preprocessing module in the SMT pipeline has been long studied, for instance in rich morphology languages such as Arabic (Habash and Sadat, 2006) or agglutinative languages such as Turkish (Bisazza and Federico, 2009) (Yeniterzi and Oflazer, 2010), and many more. This paper shows an example on how to use Indonesian morphological information on an Indonesian-English SMT system by prepro- cessing to gain better translation quality. Indonesian has a complex morphology system, in- cluding affixation, reduplication, and cliticization. Here we address the problem of cliticized phrase constructions in Indonesian that occur more frequent in spoken language and social media text than in the formal written text. Having more cliticized phrases in a text makes a spoken dialogue text difficult to translate. Here we also evaluate the effect of the pre- processing on other different types of text. 2 Related Work Indonesian or Bahasa Indonesia (“language of In- donesia”), is the official language of the country. In- donesian is the fourth most spoken language in the world with approximately 230 million speakers in- cluding its 30 million native speakers. In spite of that fact, Indonesian is an under-resourced language within the Austronesian language family. There is still a lot of work that is needed to be done to collect language resources or to build language tools for this language. Given the lack of language resources, the research on Indonesian Machine Translation (MT) is not so prolific, although MT is one of the major research topics in NLP. Related MT research is mostly done …

Content truncated to 500 words