Affective Retrofitted Word Embeddings

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 550–561 November 20–23, 2022. ©2022 Association for Computational Linguistics 550 Affective Retrofitted Word Embeddings Sapan Shah1,2, Sreedhar Reddy1, and Pushpak Bhattacharyya2 1TCS Research, Tata Consultancy Services, Pune 2Indian Institute of Technology Bombay, Mumbai {sapan.hs,sreedhar.reddy}@tcs.com pb@cse.iitb.ac.in Abstract Word embeddings learned using the distri- butional hypothesis (e.g., GloVe, Word2vec) do not capture the affective dimensions of valence, arousal, and dominance, which are present inherently in words. We present a novel retrofitting method for updating embed- dings of words for their affective meaning. It learns a non-linear transformation function that maps pre-trained embeddings to an affec- tive vector space, in a representation learn- ing setting. We investigate word embeddings for their capacity to cluster emotion-bearing words. The affective embeddings learned by our method achieve better inter-cluster and intra-cluster distance for words having the same emotions, as evaluated through differ- ent cluster quality metrics. For the down- stream tasks on sentiment analysis and sar- casm detection, simple classification models, viz. SVM and Attention Net, learned using our affective embeddings perform better than their pre-trained counterparts (more than 1.5% im- provement in F1-score) and other benchmarks. Furthermore, the difference in performance is more pronounced in limited data setting. 1 Introduction Affect refers to the experience of a feeling or emo- tion (Picard, 2000). This definition broadly en- compasses sentiment, emotion, personality, and mood. Incorporating these affective aspects in text analysis can significantly benefit numerous NLP applications, including sentiment analysis, sarcasm detection, opinion mining, empathetic agents, etc. Words, being the smallest meaningful constructs in a language, have been the primary focus area for af- fect analysis in literature. The affective meaning of a word can be represented primarily using: (1) dis- crete affective labels such as joy, happiness, anger, etc., notable models include Plutchik’s Wheel of Emotions (Plutchik, 1980), Ekman’s model (Ek- man, 1992), etc.; (2) dimensional models such as valence-arousal-dominance (VAD) model (Russell and Mehrabian, 1977), evaluation-potency-activity (EPA) model (Osgood et al., 1957), etc. that repre- sent human affects in a continuous space. In this work, we focus on dimensional models since they capture more fine-grained information compared to the discrete models and are more expressive (Calvo and Mac Kim, 2013). The dimensional model in VAD represents a word and its affective meaning as a point in a 3-dimensional space that consists of valence (degree of pleasure or displeasure), arousal (degree of excitement or calmness), and dominance (degree of control or submission). While pre-trained embeddings are good at cap- turing various lexico-semantic relations, do they encode the affective meaning of words? For exam- ple, consider violate, a word having low valence and high arousal. Table 1 shows the most similar words to violate as computed using cosine similar- ity with pre-trained Word2vec embeddings. This list includes words with high valence (e.g., comply and obey) as well as low arousal (e.g., adhere, stipu- late), disregarding the …

Content truncated to 500 words