Generating Diverse Translations with Sentence Codes

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1823–1827 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1823 Generating Diverse Translations with Sentence Codes Raphael Shu The University of Tokyo shu@nlab.ci.i.u-tokyo.ac.jp Hideki Nakayama The University of Tokyo nakayama@ci.i.u-tokyo.ac.jp Kyunghyun Cho New York University CIFAR Azrieli Global Scholar kyunghyun.cho@nyu.edu Abstract Users of machine translation systems may de- sire to obtain multiple candidates translated in different ways. In this work, we attempt to obtain diverse translations by using sen- tence codes to condition the sentence gener- ation. We describe two methods to extract the codes, either with or without the help of syntax information. For diverse generation, we sample multiple candidates, each of which conditioned on a unique code. Experiments show that the sampled translations have much higher diversity scores when using reasonable sentence codes, where the translation quality is still on par with the baselines even under strong constraint imposed by the codes. In qualitative analysis, we show that our method is able to generate paraphrase translations with drastically different structures. The proposed approach can be easily adopted to existing translation systems as no modification to the model is required. 1 Introduction When using machine translation systems, users may desire to see different candidate translations other than the best one. In this scenario, users usu- ally expect the system to show candidates with dif- ferent sentence structures. To obtain diverse translations, conventional neural machine translation (NMT) models allow one to sample translations using the beam search algorithm, however, they usually share similar sentence structures. Recently, various methods (Li et al., 2016; Xu et al., 2018) are proposed for di- verse generation. These methods encourage the model to use creative vocabulary to achieve high diversity. Although producing creative words ben- efits tasks in the dialog domain, when applied to machine translation, it can hurt the translation quality by changing the original meaning. In this work, we are interested in generating multiple valid translations with high diversity. To achieve this, we propose to construct the codes based on semantics-level or syntax-level informa- tion of target-side sentences. To generate diverse translations, we constrain the generation model by specifying a particular code as a semantic or syntactic assignment. More concretely, we prefix the target-side sentences with the codes. Then, an NMT model is trained with the original source sentences and the prefixed target sentences. As the model generates tokens in left-to-right order, the probability of emitting each word is predicted conditioned on the assigned code. As each assignment is supposed to corre- spond to a sentence structure, the candidate trans- lations sampled with different assignments are ex- pected to have high diversity. We can think such model as a mixture-of-expert translation model where each expert is capable of producing translations with a certain style indi- cated by the code. In the inference time, code as- signments are given to the model so that a selec- tion of experts are picked to generate translations. The key …

Content truncated to 500 words