Productive Generation of Compound Words in Statistical Machine Translation

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 250–260, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics Productive Generation of Compound Words in Statistical Machine Translation Sara Stymne Link¨oping University Link¨oping, Sweden sara.stymne@liu.se Nicola Cancedda Xerox Research Centre Europe Meylan, France nicola.cancedda@xrce.xerox.com Abstract In many languages the use of compound words is very productive. A common practice to reduce sparsity consists in splitting com- pounds in the training data. When this is done, the system incurs the risk of translating com- ponents in non-consecutive positions, or in the wrong order. Furthermore, a post-processing step of compound merging is required to re- construct compound words in the output. We present a method for increasing the chances that components that should be merged are translated into contiguous positions and in the right order. We also propose new heuristic methods for merging components that outper- form all known methods, and a learning-based method that has similar accuracy as the heuris- tic method, is better at producing novel com- pounds, and can operate with no background linguistic resources. 1 Introduction In many languages including most of the Germanic (German, Swedish etc.) and Uralic (Finnish, Hun- garian etc.) language families so-called closed com- pounds are used productively. Closed compounds are written as single words without spaces or other word boundaries, as the Swedish: gatstenshuggare gata + sten + huggare paving stone cutter street stone cutter To cope with the productivity of the phenomenon, any effective strategy should be able to correctly process compounds that have never been seen in the training data as such, although possibly their com- ponents have, either in isolation or within a different compound. The extended use of compounds make them prob- lematic for machine translation. For translation into a compounding language, often fewer compounds than in normal texts are produced. This can be due to the fact that the desired compounds are missing in the training data, or that they have not been aligned correctly. When a compound is the idiomatic word choice in the translation, a MT system can often produce separate words, genitive or other alternative constructions, or translate only one part of the com- pound. Most research on compound translation in com- bination with SMT has been focused on transla- tion from a compounding language, into a non- compounding one, typically into English. A com- mon strategy then consists in splitting compounds into their components prior to training and transla- tion. Only few have investigated translation into a com- pounding language. For translation into a com- pounding language, the process becomes: • Splitting compounds on the target (compound- ing language) side of the training corpus; • Learn a translation model from this split train- ing corpus from source (e.g. English) into decomposed-target (e.g. decomposed-German) • At translation time, translate using the learned model from source into decomposed-target. • Apply a post-processing “merge” step to recon- struct compounds. The merging step must solve two problems: identify which words should be merged into compounds, …

Content truncated to 500 words