Processing of Swedish compounds for phrase-based statistical machine translation

Processing of Swedish Compounds for Phrase-Based Statistical Machine Translation Sara Stymne and Maria Holmqvist Department of Computer and Information Science Link¨oping University, Sweden {sarst,marho}@ida.liu.se Abstract. We investigated the effects of processing Swedish compounds for phrase-based SMT between Swedish and English. Compounds were split in a pre-processing step using an unsupervised empirical method. After translation into Swedish, compounds were merged, using a novel merging algorithm. We investigated two ways of handling compound parts, by marking them as compound parts or by normalizing them to a canonical form. We found that compound splitting did improve transla- tion into Swedish, according to automatic metrics. For translation into English the results were not consistent across automatic metrics. How- ever, error analysis of compound translation showed a small improvement in the systems that used splitting. The number of untranslated words in the English output was reduced by 50%. 1 Introduction In many languages, including the Germanic languages, compounding is very common, and compounds are written without spaces or other word boundaries. This is problematic for many NLP applications. For phrase-based statistical machine translation (PBSMT) it leads to problems due to data sparseness, with a large number of out-of-vocabulary compounds. This problem has been studied in several papers for translation between Ger- man and English. Koehn and Knight [1] suggested an empirical algorithm for splitting compounds, which was successfully applied to German-to-English trans- lation. The same method was used by Popovi´c et al. [2] for translation in both directions between English and German. In addition, they merged compounds in a postprocessing step for translation into German. Stymne [3] tried a number of variations of the algorithm for translation in both translation directions. In both studies translation quality was improved. Compound parts are usually treated as ordinary words in the training data, e.g. in [1, 2]. In [3, 4], however, compound parts were marked with a symbol, to separate them from normal words, resulting in improved translation quality compared to an unsplit baseline. Virpioja et al.[5] used an unsupervised algorithm for morphological splitting and merging, where both compounds and other words were split into stems and affixes, for translation between Swedish and other Scandinavian languages. 12th EAMT conference, 22-23 September 2008, Hamburg, Gemany 182 Table 1. Compound forms in Swedish Type Suffixes Example None riskkapital (risk + kapital) risk capital Additions -s -t frihetsl¨angtan (frihet + l¨angtan) longing for peace Truncations -e -a pojkv¨an (pojke + v¨an) boyfriend Combinations -a/-s -a/-e -a/-u -a/-o -e/-a -e/-s -el/-la -la/-el -ra/-er arbetsgrupp (arbete + grupp) working group All but the last part of a word were marked with a symbol, which was used in the merging step. They did not get improved translations when measured automatically, but saw other advantages such as a reduction of out-of-vocabulary words. There have been many other suggestions of how to split compounds, but they are often only evaluated against a gold standard, not on a translation task. Alfonseca et al. [6] suggested a language independent supervised learning method, which needs a corpus of annotated …

Content truncated to 500 words