Semantic Structural Decomposition for Neural Machine Translation

Proceedings of the Ninth Joint Conference on Lexical and Computational Semantics (*SEM), pages 50–57 Barcelona, Spain (Online), December 12–13, 2020 50 Semantic Structural Decomposition for Neural Machine Translation Elior Sulem∗ Dept. of Computer and Information Science University of Pennsylvania eliors@seas.upenn.edu Omri Abend Ari Rappoport Dept. of Computer Science The Hebrew University of Jerusalem oabend|arir@cs.huji.ac.il Abstract Building on recent advances in semantic pars- ing and text simplification, we investigate the use of semantic splitting of the source sen- tence as preprocessing for machine translation. We experiment with a Transformer model and evaluate using large-scale crowd-sourcing ex- periments. Results show a significant increase in fluency on long sentences on an English-to- French setting with a training corpus of 5M sentence pairs, while retaining comparable ad- equacy. We also perform a manual analysis which explores the tradeoff between adequacy and fluency in the case where all sentence lengths are considered.1 1 Introduction In this paper, we apply a semantic decomposition approach for Neural Machine Translation (NMT) and demonstrate that it can tackle two of the main limitations of state-of-the-art NMT. The first is the translation of long sentences, which is a re- current issue arising in NMT evaluation (Sutskever et al., 2014; Cho et al., 2014; Pouget-Abadie et al., 2014; Su et al., 2018; Currey and Heafield, 2018). The second limitation is that current research in NMT mostly focuses on translating single sen- tences to single sentences, and is evaluated accord- ingly. However, Li and Nenkova (2015) showed that using several sentences to translate a source sentence is sometimes the preferable option. There- fore, the simplicity of the output could be an im- portant quality marker for translation. In our model, each source sentence is split (or decomposed) into semantic units, namely scenes, ∗ This work was done when being affiliated to the Hebrew University of Jerusalem 1The code and the evaluation data are avail- able at https://github.com/eliorsulem/ Semantic-Structural-Decomposition-for-NMT This work is licensed under a Creative Commons Attri- bution 4.0 International License. License details: http: //creativecommons.org/licenses/by/4.0/. building on the Direct Semantic Splitting algorithm (DSS; Sulem et al., 2018b) that uses the Universal Conceptual Cognitive Annotation (UCCA; Abend and Rappoport, 2013) scheme for semantic repre- sentation. Scenes are then translated separately and concatenated for generating the final translation output, which may consist of several sentences. Our main experiments use the state-of-the-art Transformer model (Vaswani et al., 2017) in English-to-French settings. We also include ex- periments with other MT architectures and training set sizes, and evaluate our results using the crowd- sourcing protocol of Graham et al. (2016) (§4). We obtain a significant increase in fluency on sentences longer than 30 words on the newstest2014 test cor- pus for English-to-French translation, with a train- ing corpus of 5M sentence pairs, without degrading adequacy. Considering all sentence lengths, we ob- serve a tradeoff between fluency and adequacy. We explore it using a manual analysis, suggesting that the decrease in adequacy is partly due to the loss of cohesion resulting from the splitting (§6). We then proceed …

Content truncated to 500 words