Perception vs. Acceptability of TM and SMT Output: What do translators prefer?

Compositional Source Word Representations for Neural Machine Translation Duygu Ataman FBK, Trento, Italy University of Trento, Italy ataman@fbk.eu Mattia A. Di Gangi FBK, Trento, Italy University of Trento, Italy digangi@fbk.eu Marcello Federico MMT Srl, Trento, Italy FBK, Trento, Italy federico@fbk.eu Abstract The requirement for neural machine trans- lation (NMT) models to use fixed-size in- put and output vocabularies plays an im- portant role for their accuracy and gener- alization capability. The conventional ap- proach to cope with this limitation is per- forming translation based on a vocabulary of sub-word units that are predicted us- ing statistical word segmentation methods. However, these methods have recently shown to be prone to morphological errors, which lead to inaccurate translations. In this paper, we extend the source-language embedding layer of the NMT model with a bi-directional recurrent neural network that generates compositional representa- tions of the source words from embeddings of character n-grams. Our model con- sistently outperforms conventional NMT with sub-word units on four translation di- rections with varying degrees of morpho- logical complexity and data sparseness on the source side. 1 Introduction Neural machine translation (NMT) has improved the state-of-the-art performance in machine trans- lation of many languages (Bentivogli et al., 2016; Junczys-Dowmunt et al., 2016). However, current NMT systems still suffer from poor performance in translating infrequent or unseen words, limiting their deployment for translating low-resource and morphologically-rich languages. This problem is c⃝ 2018 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. mainly caused by the fundamental design of the model, which requires observing many examples of a word until its input representation (i.e. embed- ding) becomes effective. Moreover, the convention of limiting the input and output vocabularies to few tens of thousands of words to control the compu- tational complexity of the model leads to coverage issues. In fact, a word can be translated only if an exact match of it is found in the vocabulary. To cope with this well-known problem, several studies have suggested to redefine a new model vocabulary in terms of the interior orthographic units compounding the words, such as charac- ter n-grams (Costa-Jussa and Fonollosa, 2016; Lee et al., 2016; Luong and Manning, 2016) or statistically-learned sub-word units (Sennrich et al., 2016; Wu et al., 2016; Ataman et al., 2017). In spite of providing an ideal open vocabulary so- lution, the former set of approaches mostly failed to achieve competitive results. This might be re- lated to the semantic ambiguity caused by solely relying on embeddings of character n-grams which are generally learned by disregarding any lexical context, hence, morphology. In fact, building a vo- cabulary of sub-word units for training the NMT model and performing translation based on sub- word embeddings has now become the prominent approach. However, many studies have shown that statistical word segmentation methods can break the morphological structure of words, leading to loss of semantic and syntactic information in the sentence and, consequently, inaccurate translations (Niehues et al., 2016; Ataman …

Content truncated to 500 words