Handling Homographs in Neural Machine Translation

Proceedings of NAACL-HLT 2018, pages 1336–1345 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Handling Homographs in Neural Machine Translation Frederick Liu∗†, Han Lu∗‡, Graham Neubig Language Technologies Institute Carnegie Mellon University {fliu1,hlu2,gneubig}@cs.cmu.edu Abstract Homographs, words with different meanings but the same surface form, have long caused difficulty for machine translation systems, as it is difficult to select the correct translation based on the context. However, with the ad- vent of neural machine translation (NMT) sys- tems, which can theoretically take into ac- count global sentential context, one may hy- pothesize that this problem has been allevi- ated. In this paper, we first provide empir- ical evidence that existing NMT systems in fact still have significant problems in properly translating ambiguous words. We then pro- ceed to describe methods, inspired by the word sense disambiguation literature, that model the context of the input word with context-aware word embeddings that help to differentiate the word sense before feeding it into the en- coder. Experiments on three language pairs demonstrate that such models improve the per- formance of NMT systems both in terms of BLEU score and in the accuracy of translating homographs.1 1 Introduction Neural machine translation (NMT; Sutskever et al. (2014); Bahdanau et al. (2015), §2), a method for MT that performs translation in an end-to- end fashion using neural networks, is quickly be- coming the de-facto standard in MT applications due to its impressive empirical results. One of the drivers behind these results is the ability of NMT to capture long-distance context using re- current neural networks in both the encoder, which takes the input and turns it into a continuous-space representation, and the decoder, which tracks the ∗Equal contribution. †Now at Snap Inc. ‡Now at Google 1Code for our translation models is available at https://goo.gl/oaiqoT Source Charges against four other men were found not proven . �������������������(accuse) Reference ����������������(fee) Baseline Source The couch takes up a lot of room . Reference Le canapé prend beaucoup de place . (space) Baseline Le canapé lit beaucoup de chambre . (bedroom) �����������������(accuse) Le canapé prend beaucoup de place . (space) Our Model Our Model Figure 1: Homographs where the baseline system makes mistakes (red words) but our proposed system incorporating a more direct representation of context achieves the correct translation (blue words). Defini- tions of corresponding blue and red words are in paren- thesis. target-sentence state, deciding which word to out- put next. As a result of this ability to capture long-distance dependencies, NMT has achieved great improvements in a number of areas that have bedeviled traditional methods such as phrase- based MT (PBMT; Koehn et al. (2003)), including agreement and long-distance syntactic dependen- cies (Neubig et al., 2015; Bentivogli et al., 2016). One other phenomenon that was poorly handled by PBMT was homographs – words that have the same surface form but multiple senses. As a result, PBMT systems required specific separate mod- ules to incorporate long-term context, performing word-sense (Carpuat and Wu, 2007b; Pu …

Content truncated to 500 words