Automatic Translation for Multiple NLP tasks: a Multi-task Approach to Machine-oriented NMT Adaptation

Automatic Translation for Multiple NLP tasks: a Multi-task Approach to Machine-oriented NMT Adaptation Amirhossein Tebbifakhr FBK, Trento, Italy University of Trento, Italy atebbifakhr@fbk.eu Matteo Negri FBK, Trento, Italy negri@fbk.eu Marco Turchi FBK, Trento, Italy turchi@fbk.eu Abstract Although machine translation (MT) tra- ditionally pursues “human-oriented” ob- jectives, humans are not the only possi- ble consumers of MT output. For in- stance, when automatic translations are used to feed downstream Natural Lan- guage Processing (NLP) components in cross-lingual settings, the translated texts should ideally pursue “machine-oriented” objectives that maximize the performance of these components. Tebbifakhr et al. (2019) recently proposed a reinforcement learning approach to adapt a generic neu- ral MT (NMT) system by exploiting the re- ward from a downstream sentiment classi- fier. But what if the downstream NLP tasks to serve are more than one? How to avoid the costs of adapting and maintaining one dedicated NMT system for each task? We address this problem by proposing a multi- task approach to machine-oriented NMT adaptation, which is capable to serve mul- tiple downstream tasks with a single sys- tem. Through experiments with Spanish and Italian data covering three different tasks, we show that our approach can out- perform a generic NMT system, and com- pete with single-task models in most of the settings. 1 Introduction Neural Machine Translation (NMT) systems are typically developed considering humans as the c⃝ 2020 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. end-users, and are hence optimized pursuing human-oriented requirements about the output quality. To meet these requirements, supervised NMT models are trained to maximize the probabil- ity of the given parallel corpora (Bahdanau et al., 2015; Sutskever et al., 2014), which embed the ad- equacy and fluency criteria essential for the human comprehension of a translated sentence. In an- other line of research, these objectives are directly addressed in Reinforcement Learning (Ranzato et al., 2016; Shen et al., 2016) and Bandit Learning (Kreutzer et al., 2017; Nguyen et al., 2017), where model optimization is driven by the human feed- back obtained for each translation hypothesis. However, humans are not the only possible con- sumers of MT output. In a variety of application scenarios, MT can in fact act as a pre-processor to perform other natural language processing (NLP) tasks. For instance, this is the case of text clas- sification tasks for which, in low-resource con- ditions, the paucity of training data provides a strong motivation for exploiting translation-based solutions. In tasks like sentiment classification, hate speech detection or document classification (the three application scenarios addressed in this paper) a translation-based approach would allow: i) translating the input text data from an under- resourced language into a resource-rich target lan- guage for which high-performance NLP compo- nents are available, ii) run a classifier on the trans- lated text and, finally, iii) project the results back to the original language. This approach represents a straightforward so- lution in low/medium-resource1 language settings 1Jain et al. (2019) consider …

Content truncated to 500 words