A Systematic Exploration of Diversity in Machine Translation

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1100–1111, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics A Systematic Exploration of Diversity in Machine Translation Kevin Gimpel∗ Dhruv Batra† Chris Dyer‡ Gregory Shakhnarovich∗ ∗Toyota Technological Institute at Chicago, Chicago, IL 60637, USA †Virginia Tech, Blacksburg, VA 24061, USA ‡Carnegie Mellon University, Pittsburgh, PA 15213, USA Corresponding author: kgimpel@ttic.edu Abstract This paper addresses the problem of produc- ing a diverse set of plausible translations. We present a simple procedure that can be used with any statistical machine translation (MT) system. We explore three ways of using di- verse translations: (1) system combination, (2) discriminative reranking with rich features, and (3) a novel post-editing scenario in which multiple translations are presented to users. We find that diversity can improve perfor- mance on these tasks, especially for sentences that are difficult for MT. 1 Introduction From the perspective of user interaction, the ideal machine translator is an agent that reads documents in one language and produces accurate, high qual- ity translations in another. This interaction ideal has been implicit in machine translation (MT) re- search since the field’s inception. It is the way we interact with commercial MT services (such as Google Translate and Microsoft Translator), and the way MT systems are evaluated (Bojar et al., 2013). Unfortunately, when a real, imperfect MT system makes an error, the user is left trying to guess what the original sentence means. Multiple Hypotheses. In contrast, when we look at the way other computer systems consume out- put from MT systems (or similarly unreliable tools), we see a different pattern. In a pipeline setting it is commonplace to propagate not just a single- best output but the M-best hypotheses (Venugopal et al., 2008). Multiple solutions are also used for reranking (Collins, 2000; Shen and Joshi, 2003; Collins and Koo, 2005; Charniak and Johnson, 2005), tuning (Och, 2003), minimum Bayes risk de- coding (Kumar and Byrne, 2004), and system com- bination (Rosti et al., 2007). When dealing with error-prone systems, knowing about alternatives has benefits over relying on only a single output (Finkel et al., 2006; Dyer, 2010). Need for Diversity. Unfortunately, M-best lists are a poor surrogate for structured output spaces (Finkel et al., 2006; Huang, 2008). In MT, for exam- ple, many translations on M-best lists are extremely similar, often differing only by a single punctua- tion mark or minor morphological variation. Re- cent work has explored reasoning about sets using packed representations such as lattices and hyper- graphs (Macherey et al., 2008; Tromble et al., 2008; Kumar et al., 2009), or sampling translations propor- tional to their probability (Chatterjee and Cancedda, 2010). We argue that the implicit goal behind these techniques is to better explore the output space by introducing diversity into the surrogate set. Overview and Contributions. In this work, we el- evate diversity to a first-class status and directly ad- dress the problem of generating a set of diverse, plausible translations. We use the …

Content truncated to 500 words