Multi-paraphrase Augmentation to Leverage Neural Caption Translation

Multi-paraphrase Augmentation to Leverage Neural Caption Translation Johanes Effendi1, Sakriani Sakti1,2, Katsuhito Sudoh1,2, Satoshi Nakamura1,2 1Nara Institute of Science and Technology, Japan 2RIKEN, Center for Advanced Intelligence Project AIP, Japan {johanes.effendi.ix4,ssakti,sudoh,s-nakamura}@is.naist.jp Abstract Paraphrasing has been proven to improve translation quality in machine translation (MT) and has been widely studied alongside with the development of statistical MT (SMT). In this paper, we investigate and utilize neural para- phrasing to improve translation quality in neural MT (NMT), which has not yet been much explored. Our first contribution is to propose a new way of creating a multi-paraphrase cor- pus through visual description. After that, we also proposed to construct neural paraphrase models which initiate expert models and utilize them to leverage NMT. Here, we diffuse the image information by using image-based paraphrasing without using the image itself. Our proposed image-based multi-paraphrase augmentation strategies showed improve- ment against a vanilla NMT baseline. 1. Introduction In general, sentence paraphrasing is a way to restate a con- cept with different vocabulary, style, and level of detail. As defined by De Beaugrande and Dressler, a paraphrase is an approximate conceptual equivalence among outwardly different material [1]. In many language generation tasks, paraphrasing plays a critical role for enrichment and adding flexibility. In the MT system, paraphrases are often used for multi-reference evaluation [1], pre-editing of source sen- tences [2, 3, 4] and automatic post-editing [5, 6, 7]. Moreover, since the development of SMT, there have been a lot of approaches for using paraphrasing to elaborate the source language data. Such method have been concluded as a convenient way to handle out-of-vocabulary (OOV) and rare words problem [8]. A study by Madnani and Dorr also showed that by using targeted paraphrases, unfair penaliza- tion of translation hypotheses could be avoided [9]. Para- phrasing could also be used to augment the dataset size, which correlates positively with translation result in SMT [4, 10]. However, despite a wide range of existing works of para- phrasing, MT studies usually use a strict definition of para- phrase which accepts only word substitution and reorder- ing. The reason is that we cannot grasp a tangible concept about the idea of the sentence being translated. On the other hand, Hirst argues that paraphrases don’t necessarily need to be fully synonymous. It is sufficient for them to be quasi- synonymous, as a mutually replaceable form of truth appli- cable in some contexts [11]. By taking further this idea, as long as the semantics of the mutual paraphrase sentence can be determined, we actually can widen the paraphrase defini- tion to some extent. In this research, we treat an image as a symbolic form of sentence idea, regarded as the basis of paraphrasing. We con- sider two sentences as paraphrase as long as both of them are talking about the same image. This means that the word or phrase insertion and deletion based on the same picture as a concept may now be accepted as one of the paraphrase vari- ations. Slightly different from the …

Content truncated to 500 words