Affect-LM: A Neural Language Model for Customizable Affective Text Generation

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 634–642 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1059 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 634–642 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1059 Affect-LM: A Neural Language Model for Customizable Affective Text Generation Sayan Ghosh1, Mathieu Chollet1, Eugene Laksana1, Louis-Philippe Morency2 and Stefan Scherer1 1Institute for Creative Technologies, University of Southern California, CA, USA 2Language Technologies Institute, Carnegie Mellon University, PA, USA 1{sghosh,chollet,elaksana,scherer}@ict.usc.edu 2morency@cs.cmu.edu Abstract Human verbal communication includes affective messages which are conveyed through use of emotionally colored words. There has been a lot of research in this direction but the problem of integrat- ing state-of-the-art neural language mod- els with affective information remains an area ripe for exploration. In this paper, we propose an extension to an LSTM (Long Short-Term Memory) lan- guage model for generating conversa- tional text, conditioned on affect cate- gories. Our proposed model, Affect-LM enables us to customize the degree of emotional content in generated sentences through an additional design parameter. Perception studies conducted using Ama- zon Mechanical Turk show that Affect- LM generates naturally looking emotional sentences without sacrificing grammatical correctness. Affect-LM also learns affect- discriminative word representations, and perplexity experiments show that addi- tional affective information in conversa- tional text can improve language model prediction. 1 Introduction Affect is a term that subsumes emotion and longer term constructs such as mood and personality and refers to the experience of feeling or emo- tion (Scherer et al., 2010). Picard (1997) provides a detailed discussion of the importance of affect analysis in human communication and interaction. Within this context the analysis of human affect from text is an important topic in natural language understanding, examples of which include senti- ment analysis from Twitter (Nakov et al., 2016), affect analysis from poetry (Kao and Jurafsky, Affect-LM “I feel so …” Context Words “… great about this.” “… good about this.” “… awesome about this.” Affect Strength Low High Mid Affect Category ! " # $ % Automatic Inference (optional) et−1 ct−1 β Figure 1: Affect-LM is capable of generating emotionally colored conversational text in five specific affect categories (et−1) with varying affect strengths (β). Three generated example sentences for happy affect category are shown in three distinct affect strengths. 2012) and studies of correlation between function words and social/psychological processes (Pen- nebaker, 2011). People exchange verbal messages which not only contain syntactic information, but also information conveying their mental and emo- tional states. Examples include the use of emo- tionally colored words (such as furious and joy) and swear words. The automated processing of affect in human verbal communication is of great importance to understanding spoken language sys- tems, particularly for emerging applications such as dialogue systems and conversational agents. Statistical language modeling is an integral component of speech recognition systems, with other applications such as machine translation and information retrieval. There …

Content truncated to 500 words