Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation

Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation Xiao Pu∗ Nuance Communications xiao.pu@nuance.com Nikolaos Pappas Idiap Research Institute nikolaos.pappas@idiap.ch James Henderson Idiap Research Institute james.henderson@idiap.ch Andrei Popescu-Belis HEIG-VD / HES-SO andrei.popescu-belis@heig-vd.ch Abstract This paper demonstrates that word sense disambiguation (WSD) can improve neural machine translation (NMT) by widening the source context considered when modeling the senses of potentially ambiguous words. We first introduce three adaptive cluster- ing algorithms for WSD, based on k-means, Chinese restaurant processes, and random walks, which are then applied to large word contexts represented in a low-rank space and evaluated on SemEval shared-task data. We then learn word vectors jointly with sense vectors defined by our best WSD method, within a state-of-the-art NMT sys- tem. We show that the concatenation of these vectors, and the use of a sense selec- tion mechanism based on the weighted av- erage of sense vectors, outperforms several baselines including sense-aware ones. This is demonstrated by translation on five lan- guage pairs. The improvements are more than 1 BLEU point over strong NMT base- lines, +4% accuracy over all ambiguous nouns and verbs, or +20% when scored manually over several challenging words. 1 Introduction The correct translation of polysemous words re- mains a challenge for machine translation (MT). Although some translation options may be in- terchangeable, substantially different senses of ∗ Work conducted while at the Idiap Research Institute. source words must generally be rendered by dif- ferent words in the target language. Hence, an MT system should identify—implicitly or explicitly— the correct sense conveyed by each occurrence in order to generate an appropriate translation. For instance, in the following sentence from Europarl, the translation of “deal” should convey the sense “to handle” (in French traiter) and not “to cope” (in French remédier, which is wrong): Source: How can we guarantee the system of prior notification for high-risk products at ports that have the necessary facilities to deal with them? Reference translation: Comment pouvons-nous garantir le système de notification préalable pour les produits présentant un risque élevé dans les ports qui disposent des installations nécessaires pour traiter ces produits ? Baseline neural MT: [. . .] les ports qui disposent des moyens nécessaires pour y remédier ? Sense-aware neural MT: [. . .] les ports qui dis- posent des installations nécessaires pour les traiter ? Current MT systems perform word sense disam- biguation implicitly, based on co-occurring words in a rather limited context. In phrase-based statis- tical MT, the context size is related to the order of the language model (often between 3 and 5) and to the length of n-grams in the phrase table (seldom above 5). In attention-based neural MT (NMT), the context extends to the entire sentence, but 635 Transactions of the Association for Computational Linguistics, vol. 6, pp. 635–650, 2018. Action Editor: Eneko Agirre. Submission batch: 8/2018; Published 12/2018. c⃝ 2018 Association for Computational Linguistics. Distributed under a CC-BY 4.0 license. multiple word senses are not modeled explicitly. The implicit sense information captured by word …

Content truncated to 500 words