Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6191–6196 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6191 Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach Zonghan Yang♠ Yong Cheng♣ Yang Liu♠♦∗ Maosong Sun♠ ♠Institute for Artificial Intelligence State Key Laboratory of Intelligent Technology and Systems Department of Computer Science and Technology, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology ♣Google AI ♦ Beijing Advanced Innovation Center for Language Resources Abstract While neural machine translation (NMT) has achieved remarkable success, NMT systems are prone to make word omission errors. In this work, we propose a contrastive learn- ing approach to reducing word omission er- rors in NMT. The basic idea is to enable the NMT model to assign a higher probability to a ground-truth translation and a lower proba- bility to an erroneous translation, which is au- tomatically constructed from the ground-truth translation by omitting words. We design dif- ferent types of negative examples depending on the number of omitted words, word fre- quency, and part of speech. Experiments on Chinese-to-English, German-to-English, and Russian-to-English translation tasks show that our approach is effective in reducing word omission errors and achieves better translation performance than three baseline methods. 1 Introduction While neural machine translation (NMT) has achieved remarkable success (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017), there still remains a severe challenge: NMT sys- tems are prone to omit essential words on the source side, which severely deteriorate the ade- quacy of machine translation. Due to the lack of interpretability of neural networks, it is hard to ex- plain how these omission errors occur and design methods to eliminate them. Existing methods for reducing word omission errors in NMT have focused on modeling cover- age (Tu et al., 2016; Mi et al., 2016; Wu et al., 2016; Wang et al., 2016; Tu et al., 2017). The cen- tral idea is to model the fertility (i.e., the number of corresponding target words) of a source word based on attention weights to avoid word omis- sion. Although these methods prove to be effec- tive in modeling coverage for NMT, they heav- ily rely on the attention weights provided by the ∗ Corresponding author: Yang Liu RNNsearch model (Bahdanau et al., 2015). Since the attention weights between input and output are not readily available in the state-of-the-art Trans- former model (Vaswani et al., 2017), it is hard for existing methods to be directly applicable. As a result, it is important to develop model-agnostic methods for addressing the word omission prob- lem in NMT. In this paper, we propose a simple and effective contrastive learning approach to reducing word omission errors in NMT. The basic idea is to max- imize the margin between the probability of a ground-truth translation and that of an erroneous translation for a given source sentence. The er- roneous translations are automatically constructed via omitting words among the …

Content truncated to 500 words