The Transference Architecture for Automatic Post-Editing

Proceedings of the 28th International Conference on Computational Linguistics, pages 5963–5974 Barcelona, Spain (Online), December 8-13, 2020 5963 The Transference Architecture for Automatic Post-Editing Santanu Pal1, Hongfei Xu1,2, Nico Herbig2, Sudip Naskar3, Antonio Kr¨uger2, Josef van Genabith1,2 1Saarland University, Germany 2German Research Center for Artificial Intelligence (DFKI), Saarland Informatics Campus, Germany 3Jadavpur University, Kolkata, India {firstname.lastname}@uni-saarland.de {firstname.lastname}@dfki.de Abstract In automatic post-editing (APE) it makes sense to condition post-editing (pe) decisions on both the source (src) and the machine translated text (mt) as input. This has led to multi-encoder based neural APE approaches. A research challenge now is the search for architectures that best support the capture, preparation and provision of src and mt information and its integration with pe decisions. In this paper we present an efficient multi-encoder based APE model, called transference. Unlike previous approaches, it (i) uses a transformer encoder block for src, (ii) followed by a decoder block, but without masking for self-attention on mt, which effectively acts as second encoder combining src → mt, and (iii) feeds this representation into a final decoder block generating pe. Our model outperforms the best performing systems by 1 BLEU point on the WMT 2016, 2017, and 2018 English–German APE shared tasks (PBSMT and NMT). Furthermore, the results of our model on the WMT 2019 APE task using NMT data shows performance at the level of the state-of-the-art. The inference time of our model is similar to the vanilla transformer-based NMT system although our model deals with two separate encoders. We further investigate the importance of our newly introduced second encoder and find that decreasing the number of layers hurts performance, while reducing the number of layers of the decoder does not matter much. 1 Introduction Although machine translation (MT) systems are improving rapidly, the resulting translations may still require manual post-editing (PE) to achieve human-acceptable translation. Automatic post-editing (APE) is a method that aims to automatically correct errors in machine translated text before performing actual human post-editing (PE) (Knight and Chander, 1994), thereby reducing the post-editors’ workload and increasing productivity (Pal et al., 2016a). APE systems learned from human PE data serve as down- stream MT post-processing modules to improve the overall performance. APE can therefore be viewed as a 2nd-stage MT system, translating predictable error patterns in MT output to their corresponding corrections. APE training data minimally involves MT output (mt) and the human post-edited (pe) version of mt, but additionally using the source (src) has been shown to provide further benefits (Bojar et al., 2015; Bojar et al., 2016; Bojar et al., 2017). To provide awareness of errors in mt originating from src, attention mechanisms (Bahdanau et al., 2015) allow modeling of non-local dependencies in the input or output sequences, and importantly also global dependencies between them (in our case src, mt and pe). The transformer architecture (Vaswani et al., 2017) is built solely upon such attention mechanisms completely replacing recurrence and con- volutions. The transformer uses positional encoding to encode the input and output sequences, and …

Content truncated to 500 words