Reducing Disambiguation Biases in NMT by Leveraging Explicit Word Sense Information

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4824 - 4838 July 10-15, 2022 ©2022 Association for Computational Linguistics Reducing Disambiguation Biases in NMT by Leveraging Explicit Word Sense Information Niccolò Campolungo Sapienza University of Rome campolungo@di.uniroma1.it Tommaso Pasini Sapienza University of Rome pasini@di.uniroma1.it Denis Emelin University of Edinburgh, Scotland d.emelin@sms.ed.ac.uk Roberto Navigli Sapienza University of Rome navigli@diag.uniroma1.it Abstract Recent studies have shed some light on a com- mon pitfall of Neural Machine Translation (NMT) models, stemming from their strug- gle to disambiguate polysemous words with- out lapsing into their most frequently occurring senses in the training corpus. In this paper, we first provide a novel approach for automatically creating high-precision sense-annotated paral- lel corpora, and then put forward a specifically tailored fine-tuning strategy for exploiting these sense annotations during training without intro- ducing any additional requirement at inference time. The use of explicit senses proved to be beneficial to reduce the disambiguation bias of a baseline NMT model, while, at the same time, leading our system to attain higher BLEU scores than its vanilla counterpart in 3 language pairs. 1 Introduction Translating a sentence requires the underlying meaning to be captured and then expressed in the target language. Nonetheless, only little atten- tion has been devoted to studying the actual ca- pabilities of Neural Machine Translation (NMT) approaches of modeling different senses of am- biguous words, with recent work showing that sys- tems tend to be biased towards the most frequent meanings found within the training corpus (Emelin et al., 2020). This phenomenon is hard to mea- sure through classical evaluation metrics, such as the BLEU score (Papineni et al., 2002), as they often rely on word-matching heuristics that fail to capture the disambiguation capabilities of the eval- uated systems. Therefore, several efforts have been recently devoted to shed some light and create test beds (Rios Gonzales et al., 2017; Raganato et al., 2019; Emelin et al., 2020; Campolungo et al., 2022) to challenge NMT models. Results show that these models still struggle to deal with highly polyse- mous words, especially when used to express least frequent senses. For example, given the sentence “The energy comes from a distant plant.”, both Google Trans- late and DeepL disambiguate1 plant to its sense of organism when translating into Italian, and pro- duce the following incorrect sentence “L’energia proviene da una pianta lontana.”, rather than “L’energia proviene da un impianto lontano.”, where impianto is the translation for the factory meaning of plant. This suggests that, even when adequate context is provided (energy should be enough to correctly infer the right sense of plant), state-of-the-art models might still be biased to- wards the most frequent meanings found within training data. Some recent studies have explored how to lever- age explicit sense information within NMT mod- els (Rios Gonzales et al., 2017; Pu et al., 2018a; Nguyen et al., 2018). Nevertheless, including such information is not trivial for three main reasons: i) …

Content truncated to 500 words