HiTrans: A Transformer-Based Context- and Speaker-Sensitive Model for Emotion Detection in Conversations

Proceedings of the 28th International Conference on Computational Linguistics, pages 4190–4200 Barcelona, Spain (Online), December 8-13, 2020 4190 HiTrans: A Transformer-Based Context- and Speaker-Sensitive Model for Emotion Detection in Conversations Jingye Li1, Donghong Ji1, Fei Li1,∗, Meishan Zhang2, Yijiang Liu1 1Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, China 2School of New Media and Communication, Tianjin University, China {theodorelee, dhji, cslyj}@whu.edu.cn {foxlf823, mason.zms}@gmail.com Abstract Emotion detection in conversations (EDC) is to detect the emotion for each utterance in conver- sations that have multiple speakers. Different from the traditional non-conversational emotion detection, the model for EDC should be context-sensitive (e.g., understanding the whole con- versation rather than one utterance) and speaker-sensitive (e.g., understanding which utterance belongs to which speaker). In this paper, we propose a transformer-based context- and speaker- sensitive model for EDC, namely HiTrans, which consists of two hierarchical transformers. We utilize BERT as the low-level transformer to generate local utterance representations, and feed them into another high-level transformer so that utterance representations could be sensitive to the global context of the conversation. Moreover, we exploit an auxiliary task to make our model speaker-sensitive, called pairwise utterance speaker verification (PUSV), which aims to classify whether two utterances belong to the same speaker. We evaluate our model on three benchmark datasets, namely EmoryNLP, MELD and IEMOCAP. Results show that our model outperforms previous state-of-the-art models. 1 Introduction Oh�Monica,�we�are�so sorry.�[Sadness] Monica Rachel Pheobe And�then�for�forgetting�to invite�you�to�it.�[Neutral] Well�first,�for�forgetting�to throw�you�...�[Sadness] Phoebe!�Rachel!�It's Monica!�I�wonder�...�[Joy] For�what?�[Joy] You�al-you�already�had�it? [Surprise] Rachel Monica Pheobe Monica Well,�we�called�everyone in�...�[Neutral] Figure 1: An example of emotion detection in conversations. Emotion detection in conversations (EDC), whose objective is to detect the emotion for each utterance in conversations (Wen and Wan, 2014; Li et al., 2015), has received increasing attention in the natural language processing (NLP) community (Majumder et al., 2019; Zhong et al., 2019; Ghosal et al., 2019) due to its widely applications such as opinion mining (Cambria et al., 2017) and social media analy- sis (Majumder et al., 2019). As illustrated in Figure 1, there may be multiple speakers and utterances in the conversation and an EDC model needs to detect the emotion for each utterance from these speakers. Therefore, different from the traditional non-conversational emotion detection, the emotion of an utter- ance usually depends on the context of the whole conversation. In addition, the personality of a speaker ∗Corresponding author. This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http: //creativecommons.org/licenses/by/4.0/. 4191 High-level�Transformer Low-level�Transformer�(BERT) Utterance�2 Utterance�N Utterance�1 ... MLP MLP Biaffine Biaffine Biaffine ... EDC EDC PUSV PUSV PUSV Figure 2: The high-level overview of our model HiTrans. EDC represents Emotion Detection in Conver- sations. PUSV represents Pairwise Utterance Speaker Verification. may sometimes influence the emotion of an utterance. Follow previous work (Zhang et al., 2019), we call such characteristics as context sensitivity and speaker sensitivity. Previous studies for EDC can be roughly divided into two categories, namely sequence-based methods and graph-based methods. Sequence-based methods treat …

Content truncated to 500 words