Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings

Proceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 11–19 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings Annette Rios⋆ and Laura Mascarell⋆ and Rico Sennrich†⋆ ⋆Institute of Computational Linguistics, University of Zurich †School of Informatics, University of Edinburgh Abstract Word sense disambiguation is necessary in translation because different word senses often have different translations. Neural machine translation models learn differ- ent senses of words as part of an end- to-end translation task, and their capabil- ity to perform word sense disambiguation has so far not been quantified. We ex- ploit the fact that neural translation models can score arbitrary translations to design a novel cross-lingual word sense disam- biguation task that is tailored towards eval- uating neural machine translation models. We present a test set of 7,200 lexical am- biguities for German→English, and 6,700 for German→French, and report baseline results. With 70% of lexical ambiguities correctly disambiguated, we find that word sense disambiguation remains a challeng- ing problem for neural machine transla- tion, especially for rare word senses. To improve word sense disambiguation in neural machine translation, we experiment with two methods to integrate sense em- beddings. In a first approach we pass sense embeddings as additional input to the neural machine translation system. For the second experiment, we extract lexical chains based on sense embeddings from the document and integrate this informa- tion into the NMT model. While a base- line NMT system disambiguates frequent word senses quite reliably, the annotation with both sense labels and lexical chains improves the neural models’ performance on rare word senses. 1 Introduction Semantically ambiguous words present a special challenge to machine translation systems: in or- der to produce a correct sentence in the target lan- guage, the system has to decide which meaning is accurate in the given context. Errors in lexical choice can lead to wrong or even incomprehensi- ble translations. However, quantitatively assessing errors of this type is challenging, since automatic metrics such as BLEU (Papineni et al., 2002) do not provide a sufficiently detailed analysis. Several ways of evaluating lexical choice for machine translation have been proposed in pre- vious work. Cross-lingual lexical choice tasks have been created for the evaluation of word sense disambiguation (WSD) systems (Mihalcea et al., 2010; Lefever and Hoste, 2013), and have been ap- plied to the evaluation of MT systems (Carpuat, 2013). Vickrey et al. (2005) evaluate lexical choice in a blank-filling task, where the translation of an ambiguous source word is blanked from the reference translation, and an MT system is tested as to whether it can predict it. In all these tasks, a word-level translation (or set of translations) is defined as the gold label. A major problem is that an MT system will be punished for producing a synonym, paraphrase, or inflected variant of the predefined gold label. We thus propose a more constrained task where an MT system has to se- …

Content truncated to 500 words