Debiasing Word Embeddings Improves Multimodal Machine Translation

Debiasing Word Embeddings Improves Multimodal Machine Translation Tosho Hirasawa Tokyo Metropolitan University hirasawa-tosho@ed.tmu.ac.jp Mamoru Komachi Tokyo Metropolitan University komachi@tmu.ac.jp Abstract In recent years, pretrained word embed- dings have proved useful for multimodal neural machine translation (NMT) mod- els to address the shortage of available datasets. However, the integration of pretrained word embeddings has not yet been explored extensively. Further, pre- trained word embeddings in high dimen- sional spaces have been reported to suf- fer from the hubness problem. Although some debiasing techniques have been pro- posed to address this problem for other nat- ural language processing tasks, they have seldom been studied for multimodal NMT models. In this study, we examine various kinds of word embeddings and introduce two debiasing techniques for three mul- timodal NMT models and two language pairs — English–German translation and English–French translation. With our op- timal settings, the overall performance of multimodal models was improved by up to +1.62 BLEU and +1.14 METEOR for English–German translation and +1.40 BLEU and +1.13 METEOR for English– French translation. 1 Introduction In multimodal machine translation, a target sen- tence is translated from a source sentence together with related nonlinguistic information such as vi- sual information. Recently, neural machine trans- lation (NMT) has superseded traditional statisti- cal machine translation owing to the introduction c⃝ 2019 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. of the attentive encoder-decoder model, in which machine translation is treated as a sequence-to- sequence learning problem and is trained to pay attention to the source sentence while decoding (Bahdanau et al., 2015). Pretrained word embeddings are considered an important part of neural network models in many natural language processing (NLP) tasks. In the context of NMT, pretrained word embeddings have proved useful in low-resource domains (Qi et al., 2018), in which FastText (Bojanowski et al., 2017) embeddings are used to initialize the encoder and decoder of the NMT model. They provided sub- stantial overall performance improvement for low- resource language pairs. Similarly, Hirasawa et al. (2019) introduced a multimodal NMT model with embedding prediction that provided substan- tial performance improvement. However, when word embeddings are used in the k-nearest neighbor (kNN) problem, certain words appear frequently in the k-nearest neighbors for other words (Dinu et al., 2015; Faruqui et al., 2016); this is called the hubness problem in the general machine learning domain (Radovanovi´c et al., 2010). This phenomenon harms the utility of pretrained word embeddings. In the context of NMT, Rios Gonzales et al. (2017) reported that NMT models produce less-accurate translations for less-frequent words, but they are not aware of the hubness problem in word embeddings. Instead, they proposed annotating sense labels or lexical la- bels to address this problem. However, it is known to be effective to debias word embeddings based on their local bias (Hara et al., 2015) or global bias (Mu and Viswanath, 2018) for word analogy tasks, which does not require extra expensive annotations and references. In this study, …

Content truncated to 500 words