Delete, Retrieve, Generate: a Simple Approach to Sentiment and Style Transfer

Proceedings of NAACL-HLT 2018, pages 1865–1874 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer Juncen Li∗1 Robin Jia2 He He2 Percy Liang2 1 WeChat Search Application Department, Tencent 2 Computer Science Department, Stanford University juncenli@tencent.com {robinjia,hehe,pliang}@cs.stanford.edu Abstract We consider the task of text attribute transfer: transforming a sentence to alter a specific at- tribute (e.g., sentiment) while preserving its attribute-independent content (e.g., changing “screen is just the right size” to “screen is too small”). Our training data includes only sen- tences labeled with their attribute (e.g., pos- itive or negative), but not pairs of sentences that differ only in their attributes, so we must learn to disentangle attributes from attribute- independent content in an unsupervised way. Previous work using adversarial methods has struggled to produce high-quality outputs. In this paper, we propose simpler methods mo- tivated by the observation that text attributes are often marked by distinctive phrases (e.g., “too small”). Our strongest method extracts content words by deleting phrases associated with the sentence’s original attribute value, re- trieves new phrases associated with the target attribute, and uses a neural model to fluently combine these into a final output. On human evaluation, our best method generates gram- matical and appropriate responses on 22% more inputs than the best previous system, av- eraged over three attribute transfer datasets: altering sentiment of reviews on Yelp, altering sentiment of reviews on Amazon, and altering image captions to be more romantic or humor- ous. 1 Introduction The success of natural language generation (NLG) systems depends on their ability to carefully con- trol not only the topic of produced utterances, but also attributes such as sentiment and style. The de- sire for more sophisticated, controllable NLG has led to increased interest in text attribute transfer— the task of editing a sentence to alter specific at- tributes, such as style, sentiment, and tense (Hu ∗Work done while the author was a visiting researcher at Stanford University. great food but horrible staff and very very rude workers ! target=positive great food staff and very workers ! great food , awesome staff , very personable and very efficient atmosphere ! Delete attribute markers Run system (b) Attribute transfer neg pos pos pos pos neg neg neg worst very disappointed won't be back ... delicious great place for well worth ... (a) Extracting attribute markers Figure 1: An overview of our approach. (a) We identify attribute markers from an unaligned corpus. (b) We transfer attributes by removing markers of the original attribute, then generating a new sentence conditioned on the remaining words and the target attribute. et al., 2017; Shen et al., 2017; Fu et al., 2018). In each of these cases, the goal is to convert a sen- tence with one attribute (e.g., negative sentiment) to one with a different attribute (e.g., positive sen- timent), while preserving all attribute-independent content1 (e.g., what properties of a restaurant are being discussed). Typically, aligned sentences …

Content truncated to 500 words