Facilitating Multi-turn Emotional Support Conversation with Positive Emotion Elicitation: A Reinforcement Learning Approach

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1714–1729 July 9-14, 2023 ©2023 Association for Computational Linguistics Facilitating Multi-turn Emotional Support Conversation with Positive Emotion Elicitation: A Reinforcement Learning Approach Jinfeng Zhou1,2∗ † Zhuang Chen1† Bo Wang2‡ Minlie Huang1 1The CoAI Group, DCST, Institute for Artificial Intelligence, State Key Lab of Intelligent Technology and Systems, 1Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China 2College of Intelligence and Computing, Tianjin University, Tianjin, China jfzhou.mail@gmail.com, zhchen-nlp@mail.tsinghua.edu.cn, bo_wang@tju.edu.cn, aihuang@tsinghua.edu.cn Abstract Emotional support conversation (ESC) aims to provide emotional support (ES) to improve one’s mental state. Existing works stay at fitting grounded responses and responding strategies (e.g., question), which ignore the effect on ES and lack explicit goals to guide emotional posi- tive transition. To this end, we introduce a new paradigm to formalize multi-turn ESC as a pro- cess of positive emotion elicitation. Addressing this task requires finely adjusting the elicitation intensity in ES as the conversation progresses while maintaining conversational goals like co- herence. In this paper, we propose SUPPORTER, a mixture-of-expert-based reinforcement learn- ing model, and well design ES and dialogue co- herence rewards to guide policy’s learning for responding. Experiments verify the superiority of SUPPORTER in achieving positive emotion elicitation during responding while maintaining conversational goals including coherence. 1 Introduction Emotional support (ES) aims to reassure a person to recover from emotional distress and improve one’s mental state (Burleson, 2003). It is a manifesta- tion of emotional intelligence in social interactions (Heaney and Israel, 2008; Atoum and Al-Shoboul, 2018). Endowing ES into social dialogue systems for building helpful and trustful agents is an emerg- ing trend (Huang et al., 2020; Rains et al., 2020). To achieve this goal, a typical practice is mod- eling empathy, which aims to perceive and under- stand the situation and feelings of others (Keskin, 2014). Yet, the empathetic conversation (Rashkin et al., 2019) is inherently deficient in providing ES as (1) Lack of consideration of multi-turn conver- sation. Just making empathetic responses in each single dialogue turn leads to ignoring the user’s feedback and mental state changes in multi-turn ∗Work done during internship at the CoAI Group. †Equal contribution. ‡Corresponding author. My school was closed without any prior warning due to the pandemic. I understand you. I would also have been really frustrated if that happened to me. Yeah! I don't even know what is going to happen with our final. That is really upsetting and stressful. Maybe you can talk to your parents or a close friend about this, they are also frustrated due to the same experience. Yes, you are right! I am not the only one affected by the pandemic. Come on! I believe that you will definitely be able to overcome the difficulties and find a solution! Yeah, definitely! Thanks for your support, I feel better now. Figure 1: A simplified multi-turn ESC example between the user (left) and agent (right). The agent progressively adjusts the …

Content truncated to 500 words