Decode with Template: Content Preserving Sentiment Transfer

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4671–4679 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4671 Decode with Template: Content Preserving Sentiment Transfer Zhiyuan Wen†, Jiannong Cao†, Ruosong Yang†, Senzhang Wang‡ †Department of Computing, The Hong Kong Polytechnic University, ‡College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics †Kowloon, Hong Kong, China, ‡Nanjing, China {cszwen, csjcao, csryang}@comp.polyu.edu.hk, szwang@nuaa.edu.cn Abstract Sentiment transfer aims to change the underlying sentiment of input sentences. The two major challenges in existing works lie in (1) effectively disentangling the original sentiment from input sentences; and (2) preserving the semantic content while transferring the sentiment. We find that identifying the sentiment-irrelevant content from input sentences to facilitate generating output sentences could address the above challenges and then propose the Decode with Template model in this paper. We first mask the explicit sentiment words in input sentences and use the rest parts as templates to eliminate the original sentiment. Then, we input the templates and the target sentiments into our bidirectionally guided variational auto-encoder (VAE) model to generate output. In our method, the template preserves most of the semantics in input sentences, and the bidirectionally guided decoding captures both forward and backward contextual information to generate output. Both two parts contribute to better content preservation. We evaluate our method on two review datasets, Amazon and Yelp, with automatic evaluation methods and human rating. The experimental results show that our method significantly outperforms state-of-the-art models, especially in content preservation. Keywords: Text Generation, Sentiment Analysis, Bidirectionally Guided Decoding 1. Introduction Sentiment transfer for text considers the semantics of a sentence in two aspects: the sentiment information, and the content independent to sentiment information1. This task aims to change the underlying sentiment of the input text and simultaneously retain the content. As an example shown in Table 1., only the attitude to the restaurant in the review is changed, while the sentiment-independent content about the restaurant is preserved. This task requires to gen- erate sentences that (1) conform to the target sentiments, (2) preserve the semantic content of the input sentences, and (3) be fluent and readable (Jin et al., 2019). It con- nects sentiment analysis and Natural Language Generation (Zhang et al., 2018a) and facilitates a lot of NLP applica- tions such as fighting against offensive language in social media (Santos et al., 2018), news rewriting, and building controllable dialogue systems. However, this task is diffi- cult in practice due to the lack of parallel data (sentences with similar content but different sentiments). Several recent works (Shen et al., 2017; Hu et al., 2017; Yang et al., 2018; John et al., 2018) try to disentangle sentiments from content by assuming the texts are gener- ated conditioned on two latent distributional representa- tions: one with only content information, and the other with only sentiment information. Most of them focus on changing the sentiment yet fail to keep the content. The reason is that distributional disentanglement needs …

Content truncated to 500 words