Syntactic Models for Structural Word Insertion and Deletion during Translation

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 735–744, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics Syntactic Models for Structural Word Insertion and Deletion Arul Menezes and Chris Quirk Microsoft Research One Microsoft Way, Redmond, WA 98052, USA {arulm, chrisq}@microsoft.com Abstract An important problem in translation neglected by most recent statistical machine translation systems is insertion and deletion of words, such as function words, motivated by linguistic structure rather than adjacent lexical context. Phrasal and hierarchical systems can only insert or delete words in the context of a larger phrase or rule. While this may suffice when translating in-domain, it performs poorly when trying to translate broad domains such as web text. Various syntactic approaches have been proposed that begin to address this problem by learning lexicalized and unlexicalized rules. Among these, the treelet approach uses unlexicalized order templates to model ordering separately from lexical choice. We introduce an extension to the latter that allows for structural word insertion and deletion, without requiring a lexical anchor, and show that it produces gains of more than 1.0% BLEU over both phrasal and baseline treelet systems on broad domain text. 1 Introduction Among the phenomena that are modeled poorly by modern SMT systems is the insertion and deletion of words, such as function words, that are motivated by the divergent linguistic structure between source and target language. To take the simplest of examples, the English noun compound “file name” would typically be translated into Spanish as “nombre de archivo”, which requires the insertion of the preposition “de”. Conversely, when translating from Spanish to English, the “de” must be deleted. At first glance, the problem may seem trivial, yet the presence and position of these function words can have crucial impact on the adequacy and fluency of translation. In particular, function words are often used to denote key semantic information. They may be used to denote case information, in languages such as Japanese. Failing to insert the proper case marker may render a sentence unreadable or significantly change its meaning. Learning these operations can be tricky for MT models best suited to contiguous word sequences. From a fluency standpoint, proper insertion of determiners and prepositions can often make the difference between laughably awkward output and natural sounding translations; consider the output “it’s a cake piece” as opposed to “it’s a piece of cake”. Furthermore, since missing or spurious function words can confuse the target language model, handling these words properly can have an impact beyond the words themselves. This paper focuses on methods of inserting and deleting words based on syntactic cues, to be used in the context of a syntax-informed translation system. While the models we build are relatively simple and the underlying templates are easy to extract, they add significant generalization ability to the base translation system, and result in significant gains. 2 Background As a motivating example, let us return to the English/Spanish pair “file name” and “nombre de archivo”. In principle, we would …

Content truncated to 500 words