From Disjoint Sets to Parallel Data to Train Seq2Seq Models for Sentiment Transfer

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 689–698 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 689 From Disjoint Sets to Parallel Data to Train Seq2Seq Models for Sentiment Transfer Paulo Cavalin, Marisa Vasconcelos, Marcelo Grave, Claudio Pinhanez, Victor Henrique Alves Ribeiro IBM Research S˜ao Paulo, SP, Brazil pcavalin@br.ibm.com Abstract We present a method for creating parallel data to train Seq2Seq neural networks for sentiment transfer. Most systems for this task, which can be viewed as monolingual machine trans- lation (MT), have relied on unsupervised meth- ods, such as Generative Adversarial Networks (GANs)-inspired approaches, for coping with the lack of parallel corpora. Given that the literature shows that Seq2Seq methods have been consistently outperforming unsupervised methods in MT-related tasks, in this work we exploit the use of semantic similarity compu- tation for converting non-parallel data onto a parallel corpus. That allows us to train a trans- former neural network for the sentiment trans- fer task, and compare its performance against unsupervised approaches. With experiments conducted on two well-known public datasets, i.e. Yelp and Amazon, we demonstrate that the proposed methodology outperforms existing unsupervised methods very consistently in flu- ency, and presents competitive results in terms of sentiment conversion and content preserva- tion. We believe that this works opens up an opportunity for seq2seq neural networks to be better exploited in problems for which they have not been applied owing to the lack of par- allel training data. 1 Introduction Sentiment transfer can be considered as a subset of the style transfer task, the main goal of which is to convert a text that presents a style τ1 to an- other style τ2, while keeping its original meaning µ. Given the increasing number of applications that currently make use of natural language user interfaces, style transfer can be useful in many real- world applications, for instance, chatbot personality transformation for fitting chatbot language to a spe- cific public, bias removal (such as gender and racial bias), offensive and hate speech-language filtering, and thus forth. Previous studies on style transfer focused mostly on unsupervised methods, for instance Generative Adversarial Networks (GANs), owing to the lack of parallel corpora. However, given that style transfer can be viewed as a monolingual machine transla- tion (MT) task, and that seq2seq models such as the transformer have shown to outperform unsuper- vised methods in multi-lingual MT when a suffi- ciently large parallel corpus is available (Lample et al., 2018; Artetxe et al., 2019; Subramanian et al., 2018), in our opinion it is expected that seq2seq would outperform unsupervised approaches if par- allel data is available for style transfer. However, to the best of our knowledge, a parallel corpus for style transfer currently does not exist. But consid- ering that semantic similarity metrics are becom- ing more and more effective (Schwenk and Douze, 2017; Wu et al., 2018), and that considerably large non-parallel data exist for some style transfer tasks, for instance, sentiment transfer and the Yelp and Amazon review datasets, one …

Content truncated to 500 words