IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3097–3109, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3097 IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation Zhijing Jin∗ University of Hong Kong zhijing.jin@connect.hku.hk Di Jin∗ MIT jindi15@mit.edu Jonas Mueller Amazon Web Services jonasmue@amazon.com Nicholas Matthews ASAPP Inc. nmatthews@asapp.com Enrico Santus MIT esantus@csail.mit.edu Abstract Text attribute transfer aims to automatically rewrite sentences such that they possess cer- tain linguistic attributes, while simultaneously preserving their semantic content. This task remains challenging due to a lack of super- vised parallel data. Existing approaches try to explicitly disentangle content and attribute information, but this is difficult and often re- sults in poor content-preservation and ungram- maticality. In contrast, we propose a simpler approach, Iterative Matching and Translation (IMaT), which: (1) constructs a pseudo- parallel corpus by aligning a subset of se- mantically similar sentences from the source and the target corpora; (2) applies a standard sequence-to-sequence model to learn the at- tribute transfer; (3) iteratively improves the learned transfer function by refining imperfec- tions in the alignment. In sentiment modifica- tion and formality transfer tasks, our method outperforms complex state-of-the-art systems by a large margin. As an auxiliary contribu- tion, we produce a publicly-available test set with human-generated transfer references.1. 1 Introduction An intelligent natural language generation (NLG) system should be able to flexibly control linguis- tic attributes of the text it generates. For instance, a certain level of formality should be maintained in serious situations, while informal conversations can be improved through a more relaxed style. The ability to generate or rewrite a certain text with some attributes controlled or transferred to meet pragmatic goals is widely needed in applications such as dialogue generation (Oraby et al., 2018), author obfuscation (Kacmarcik and Gamon, 2006; ∗Equal Contribution 1The enriched test set is available at https:// github.com/zhijing-jin/IMaT Juola and Vescovi, 2011), and written communica- tion assistance (Pavlick and Tetreault, 2016). One typical example of a text attribute is the linguistic style, which refers to features of lexicons, syntax and phonology that collectively contribute to the identifiability of an instance of language (Gatt and Krahmer, 2018). For instance, given text written in an older style like the Shakespearean genre, a sys- tem may be tasked to convert it into modern-day colloquial English (Kabbara and Cheung, 2016). In general, a text attribute transfer system must be able to: (1) produce sentences that conform to the target attribute, (2) preserve the source sen- tence’s content, and (3) generate fluent language. Satisfying these requirements is challenging due to the typical absence of supervised parallel cor- pora exemplifying the desired attribute transfor- mations. With access to only non-parallel data, most existing approaches aim to tease apart con- tent and attribute in a latent sentence representa- tion. For instance, Shen et al. (2017) and Fu et al. (2018) utilize Generative Adversarial Networks (GANs) to learn this separation. Although these elaborate models …

Content truncated to 500 words