Improved Neural Machine Translation using Side Information

Improved Neural Machine Translation using Side Information Cong Duy Vu Hoang† and Gholamreza Haffari‡ and Trevor Cohn† † University of Melbourne, Melbourne, VIC, Australia ‡ Monash University, Clayton, VIC, Australia vhoang2@student.unimelb.edu.au, gholamreza.haffari@monash.edu, t.cohn@unimelb.edu.au Abstract In this work, we investigate whether side information is helpful in the context of neural machine translation (NMT). We study various kinds of side information, including topical information and personal traits, and then propose different ways of incorporating these information sources into existing NMT models. Our experi- mental results show the benefits of side in- formation in improving the NMT models. 1 Introduction Neural machine translation is the task of gener- ating a target language sequence given a source language sequence, framed as a neural network (Sutskever et al., 2014; Bahdanau et al., 2015, in- ter alia). Most research efforts focus on inducing more prior knowledge (Cohn et al., 2016; Zhang et al., 2017; Mi et al., 2016, inter alia), incor- porating linguistics factors (Hoang et al., 2016b; Sennrich and Haddow, 2016; Garc´ıa-Mart´ınez et al., 2017) or changing the network architecture (Gehring et al., 2017b,a; Vaswani et al., 2017; El- bayad et al., 2018) in order to better exploit the source representation. Consider a different direc- tion, situations in which there exists other modal- ity other than the text of the source sentence. For instance, the WMT 2017 campaign1 proposed to use additional information obtained from images to enrich the neural MT models, as in (Calixto et al., 2017; Matusov et al., 2017; Calixto and Liu, 2017). This task, also known as multi-modal translation, seeks to leverage images which can contain cues representing the perception of the image in source text, and potentially can con- tribute to resolve ambiguity (e.g., lexical, gender), 1http://www.statmt.org/wmt17/ multimodal-task.html vagueness, out-of-vocabulary terms, and topic rel- evancy. Inspired from the idea of multi-modal transla- tion, in our work, we propose the use of another modality, namely metadata or side information. Previously, Hoang et al. (2016a) have shown the usefulness of side information for neural language models. This work will investigate the potential usefulness of side information for NMT models. In our work, we target towards unstructured and heterogeneous side information which potentially can be found in practical applications. Specifi- cally, we investigate different kinds of side infor- mation, including topic keywords, personality in- formation and topic classification. Then we study different methods with minimal efforts for incor- porating such side information into existing NMT models. 2 Machine Translation Data with Side Information First, let’s explore some realistic scenarios in which the side information is potentially useful for NMT. TED Talks The TED Talks website2 hosts tech- nical videos from influential speakers around the world on various topics or domains, such as: ed- ucation, business, science, technology, creativity, etc. Thanks to users’ contributions, most of such videos are subtitled in multiple languages. Based on this website, Cettolo et al. (2012) created a parallel corpus for the MT research community. Inspired by this, Chen et al. (2016) further cus- tomised this dataset and included …

Content truncated to 500 words