Adapting a Language Model for Controlled Affective Text Generation

Proceedings of the 28th International Conference on Computational Linguistics, pages 2787–2801 Barcelona, Spain (Online), December 8-13, 2020 2787 Adapting a Language Model for Controlled Affective Text Generation Tushar Goswamy∗ Ishika Singh∗ Ahsan Barkati∗ Indian Institute of Technology Kanpur (IITK)Ashutosh Modi {tgoswamy, ishikas, ahsanb}@iitk.ac.in ashutoshm@cse.iitk.ac.in Abstract Human use language not just to convey information but also to express their inner feelings and mental states. In this work, we adapt the state-of-the-art language generation models to generate affective (emotional) text. We posit a model capable of generating affect-driven and topic focused sentences without losing grammatical correctness as the affect intensity increases. We propose to incorporate emotion as prior for the probabilistic state-of-the-art text generation model such as GPT-2. The model gives a user the flexibility to control the category and intensity of emotion as well as the topic of the generated text. Previous attempts at modelling fine-grained emotions fall out on grammatical correctness at extreme intensities, but our model is resilient to this and delivers robust results at all intensities. We conduct automated evaluations and human studies to test the performance of our model, and provide a detailed comparison of the results with other models. In all evaluations, our model outperforms existing affective text generation models. 1 Introduction Affect (emotion) in language plays a critical role in conveying mental and emotional states, along with the information intended to be conveyed. Machine Learning (ML) based text generation models are focused on minimising the error in the generated text by maintaining grammatical correctness, this often results in the generation of monotonous and dull conversations. Since current ML based models are trained on large corpora without any explicit affective information, they are unable to capture the emotional aspects of conversations explicitly (Asghar et al., 2018). There is a need for the automated processing and generation of affect-driven language. This will improve the quality of responses generated by a conversational system by making them more empathetic towards a human user (Colombo et al., 2019). The role of controlled emotion in the generated text is to ensure more meaningful conversations between an AI-agent and a human. It is also aimed at establishing an emotional connect with the reader of the text. Such a model can be particularly useful for conversational therapy bots for generating appropriate emotional responses based on the user’s mental state (Spring et al., 2019). Affective generation model could also be useful in the development of interactive virtual agents (Janghorbani et al., 2019). Affective advertisement generation is another area of application, especially for social advertisements aimed at inviting donations for a cause, where we need to emotionally appeal to the benefits of donation (Moran and Bagchi, 2019). Current research on incorporating language in robot learning (Luketina et al., 2019; Bisk et al., 2020) can potentially benefit not just by enabling effective human-robot dialog, but also by optimizing other reinforcement learning based components. For instance, user’s responses can be exploited to extract implicit emotion-based reward cues (Sumers et al., 2020), and the robot can respond in …

Content truncated to 500 words