FeelingBlue: A Corpus for Understanding the Emotional Connotation of Color in Context

FeelingBlue: A Corpus for Understanding the Emotional Connotation of Color in Context Amith Ananthram1 and Olivia Winn1 and Smaranda Muresan1,2 1Department of Computer Science, Columbia University, USA 2Data Science Institute, Columbia University, USA {amith,olivia,smara}@cs.columbia.edu Abstract While the link between color and emotion has been widely studied, how context-based changes in color impact the intensity of per- ceived emotions is not well understood. In this work, we present a new multimodal dataset for exploring the emotional connotation of color as mediated by line, stroke, texture, shape, and language. Our dataset, FeelingBlue, is a collection of 19,788 4-tuples of abstract art ranked by annotators according to their evoked emotions and paired with rationales for those annotations. Using this corpus, we present a baseline for a new task: Justified Af- fect Transformation. Given an image I, the task is to 1) recolor I to enhance a specified emotion e and 2) provide a textual justifica- tion for the change in e. Our model is an ensemble of deep neural networks which takes I, generates an emotionally transformed color palette p conditioned on I, applies p to I, and then justifies the color transformation in text via a visual-linguistic model. Experimental re- sults shed light on the emotional connotation of color in context, demonstrating both the promise of our approach on this challenging task and the considerable potential for future investigations enabled by our corpus.1 1 Introduction Color is a powerful tool for conveying emotion across cultures, a connection apparent in both language and art (Mohr and Jonauskaite, 2022; Mohammad and Kiritchenko, 2018). Metaphoric language frequently uses color as a vehicle for emotion: Familiar English metaphors include ‘‘feeling blue’’ or ‘‘green with envy’’. Simi- larly, artists often pick specific colors in order to convey particular emotions in their work, while viewer perceptions of a piece of art are affected by its color palette (Sartori et al., 2015). Previous 1Our dataset, code, and models are available at https:// github.com/amith-ananthram/feelingblue. studies have mostly been categorical, focusing on confirming known links between individual colors and emotions like blue & sadness or yel- low & happiness (Machajdik and Hanbury, 2010; Sartori et al., 2015; Zhang et al., 2011). However, in the wild, the emotional connotation of color is often mediated by line, stroke, texture, shape, and language. Very little work has examined these as- sociations. Does the mere presence of blue make an image feel sad? If it is made bluer, does it feel sadder? Is it dependent on its associated form or its surrounding color context? And, if the change is reflected in an accompanying textual rationale, is it more effective? Our work is the first to explore these questions. We present FeelingBlue, a new corpus of relative emotion labels for abstract art paired with English rationales for the emotion labels (see Figure 1 and Section 3). A challenge with such annotations is the extreme subjectivity inherent to emotion. In contrast to existing Likert-based corpora, we employ a Best-Worst Scaling (BWS) annotation scheme that is more consistent …

Content truncated to 500 words