EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition

Findings of the Association for Computational Linguistics: ACL 2022, pages 1610 - 1618 May 22-27, 2022 c⃝2022 Association for Computational Linguistics EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition Zaijing Li1, Fengxiao Tang1*, Ming Zhao1*, Yusen Zhu2 1School of Computer Science and Engineering, Central South University, Changsha, China 2School of Mathematics, Hunan University, Changsha, China {lizaijing,tangfengxiao,meanzhao}@csu.edu.cn zhu_yusen@163.com Abstract Emotion recognition in conversation (ERC) aims to analyze the speaker’s state and iden- tify their emotion in the conversation. Recent works in ERC focus on context modeling but ignore the representation of contextual emo- tional tendency. In order to extract multi-modal information and the emotional tendency of the utterance effectively, we propose a new struc- ture named Emoformer to extract multi-modal emotion vectors from different modalities and fuse them with sentence vector to be an emotion capsule. Furthermore, we design an end-to-end ERC model called EmoCaps, which extracts emotion vectors through the Emoformer struc- ture and obtain the emotion classification re- sults from a context analysis model. Through the experiments with two benchmark datasets, our model shows better performance than the existing state-of-the-art models. 1 Introduction Emotion recognition in conversation (ERC) is a work that recognizes the speaker’s emotion and its influencing factors in the process of conversation. Nowadays, social media such as Facebook and Twitter generate a large amount of dialogue data with various modalities of textual, audio, and video all the time. The study of speaker emotional ten- dency has huge potential value in the fields of pub- lic opinion analysis, shopping, and consumption. Therefore, conversation emotion recognition has attracted more and more attention from researchers and companies. In ERC, existing research mainly focuses on the way of contextual information modeling (Ma- jumder et al., 2019; Ghosal et al., 2019). How- ever, These models have some shortcomings due to their inability to better extract the grammatical and semantic information of the utterance. Recent studies (Yuzhao Mao et al., 2020; Weizhou Shen et al., 2020) have introduced the transformer struc- ture into the utterance feature extraction to solve Figure 1: Visualization of the heatmap for an utterance in a conversation, with three modalities. the above problems. Li et al. (2021) proposed a new expression vector, "emotion vector" for ERC, which is obtained by mapping from sentence vector, but only for textual modality. Meanwhile, existing studies (Song et al., 2004; Dellaert et al., 1996; Amir, 1998) have shown that only textual informa- tion is not enough for emotional presentation, the tone and intonation reflect the speaker’s emotions to a certain extent, and the facial expressions also express the inner feelings of the speaker in most cases. As shown in Figure 1, different modalities contain different information, and all are slightly flawed, so multi-modal based information can bet- ter identify the speaker’s emotion than a single modality in ERC. In order to identify the speaker’s emotion in con- versation effectively, it is necessary to obtain good utterance features. Also we can’t ignore the role of the utterance’s emotional tendency. As shown in …

Content truncated to 500 words