A Hierarchical VAE for Calibrating Attributes while Generating Text using Normalizing Flow

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2405–2415 August 1–6, 2021. ©2021 Association for Computational Linguistics 2405 A Hierarchical VAE for Calibrating Attributes while Generating Text using Normalizing Flow Bidisha Samanta IIT Kharagpur bidisha@iitkgp.ac.in Mohit Agrawal IIT Kharagpur mohit@iitkgp.ac.in Niloy Ganguly IIT Kharagpur niloy@cse.iitkgp.ac.in Abstract In this digital age, online users expect person- alized content. To cater to diverse group of au- diences across online platforms it is necessary to generate multiple variants of same content with differing degree of characteristics (senti- ment, style, formality, etc.). Though text-style transfer is a well explored related area, it fo- cuses on flipping the style attribute polarity instead of regulating a fine-grained attribute transfer. In this paper we propose a hierarchi- cal architecture for finer control over the at- tribute, preserving content using attribute dis- entanglement. We demonstrate the effective- ness of the generative process for two different attributes with varied complexity, namely sen- timent and formality. With extensive experi- ments and human evaluation on five real-world datasets, we show that the framework can gen- erate natural looking sentences with finer de- gree of control of intensity of a given attribute. 1 Introduction The ubiquity of online social networks and world wide web has brought in diverse and often conflict- ing groups of users consuming similar information but from different perspectives. So the onus falls on the content producer to cater customized content based on the users’ profile. Consider an exam- ple related to a Spanish football (soccer) league. Say the news is “Barcelona has defeated Real Madrid”. This news needs to be presented in differ- ent tones to a Barcelona Fan - “Barcelona smashed Real-Madrid”, a Real-Madrid Fan - “Real Madrid lost the epic battle” and a (say) Villarreal Fan - “Barcelona wins three points against Real-Madrid”. Automatic generation of content with fine regu- lation of attributes like sentiment and style is ex- tremely beneficial in this context. There are several related works in similar space of text-style-transfer techniques (Hu et al., 2017; Logeswaran et al., 2018; Shen et al., 2017; Singh and Palod, 2018) which attempt to switch polarity of a text from, e.g., formal to casual, or positive to negative senti- ment. However, none of the work focuses on more involved problem of fine-grained regulation of at- tributes to generate multiple variants of a sentence. Several of the existing style-transfer methods (Fu et al., 2018; John et al., 2018) convert a continu- ous entangled generative representation space ob- tained using variational auto-encoder (Bowman et al., 2015) into disentangled attribute and content space. It facilitates attribute polarity switch by per- turbing attribute representation without interfering with context. However, a disentangled generative representation may result in a loss of information about complex inter-dependency of content and attributes otherwise captured in an unmodified en- tangled generative space. Hence, trivial extension of the variational inference (encoding) mechanism for finer attribute control by allowing incremental perturbation of the attribute …

Content truncated to 500 words