Improving Robustness of Machine Translation with Synthetic Noise

Proceedings of NAACL-HLT 2019, pages 1916–1920 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1916 Improving Robustness of Machine Translation with Synthetic Noise Vaibhav∗, Sumeet Singh∗, Craig Stewart∗, Graham Neubig Language Technologies Institute School of Computer Science Carnegie Mellon University {vvaibhav,sumeets,cas1, gneubig}@cs.cmu.edu Abstract Modern Machine Translation (MT) systems perform remarkably well on clean, in-domain text. However most human generated text, particularly in the realm of social media, is full of typos, slang, dialect, idiolect and other noise which can have a disastrous impact on the accuracy of MT. In this paper we propose methods to enhance the robustness of MT sys- tems by emulating naturally occurring noise in otherwise clean data. Synthesizing noise in this manner we are ultimately able to make a vanilla MT system more resilient to naturally occurring noise, partially mitigating loss in ac- curacy resulting therefrom 1. 1 Introduction Machine Translation (MT) systems have been shown to exhibit severely degraded performance when required to translate of out-of-domain or noisy data (Luong and Manning, 2015; Sakaguchi et al., 2016; Belinkov and Bisk, 2017). This is particularly pronounced when systems trained on clean, formalized parallel data such as Eu- roparl (Koehn, 2005), are tasked with translation of unedited, human generated text such as is com- mon in domains such as social media, where ac- curate translation is becoming of widespread rele- vance (Michel and Neubig, 2018). Improving the robustness of MT systems to nat- urally occurring noise presents an important and interesting task. Recent work on MT robustness (Belinkov and Bisk, 2017) has demonstrated the need to build or adapt systems that are resilient to such noise. We approach the problem of adapt- ing to noisy data aiming to answer two primary research questions: ∗ These authors contributed equally 1Code available at https://github.com/ MysteryVaibhav/robust_mtnt 1. Can we artificially synthesize the types of noise common to social media text in other- wise clean data? 2. Are we able to improve the performance of vanilla MT systems on noisy data by leverag- ing artificially generated noise? In this work we present two primary meth- ods of synthesizing natural noise, in accordance with the types of noise identified in prior work as naturally occurring in internet and social me- dia based text (Eisenstein, 2013; Michel and Neu- big, 2018). Specifically, we introduce a synthetic noise induction model which heuristically intro- duces types of noise unique to social media text and labeled back translation (Sennrich et al., 2015a), a data-driven method to emulate target noise. We present a series of experiments based on the Machine Translation of Noisy Text (MTNT) data set (Michel and Neubig, 2018) through which we demonstrate improved resilience of a vanilla MT system by adaptation using artificially noised data. 2 Related Work Szegedy et al. (2013) demonstrate the fragility of neural networks to noisy input. This fragility has been shown to extend to MT systems (Belinkov and Bisk, 2017; Khayrallah and Koehn, 2018) where both artificial and natural noise are shown to negatively …

Content truncated to 500 words