BPM_MT: Enhanced Backchannel Prediction Model using Multi-Task Learning

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3447–3452 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3447 BPM_MT: Enhanced Backchannel Prediction Model using Multi-Task Learning Jin Yea Jang1,2, San Kim2, Minyoung Jung2, Saim Shin2, and Gahgene Gweon1 1Department of Intelligence and Information, Seoul National University, Republic of Korea 2AIRC, Korea Electronics Technology Institute, Republic of Korea {jinyea.jang,ggweon}@snu.ac.kr {kimsan0622,minyoung.jung,sishin}@keti.re.kr Abstract Backchannel (BC), a short reaction signal of a listener to a speaker’s utterances, helps to improve the quality of the conversation. Sev- eral studies have been conducted to predict BC in conversation; however, the utilization of ad- vanced natural language processing techniques using lexical information presented in the ut- terances of a speaker has been less considered. To address this limitation, we present a BC prediction model called BPM_MT (Backchan- nel prediction model with multitask learn- ing), which utilizes KoBERT, a pre-trained lan- guage model. The BPM_MT simultaneously carries out two tasks at learning: 1) BC cat- egory prediction using acoustic and lexical features, and 2) sentiment score prediction based on sentiment cues. BPM_MT exhibited 14.24% performance improvement compared to the existing baseline in the four BC cate- gories: continuer, understanding, empathic re- sponse, and No BC. In particular, for em- pathic response category, a performance im- provement of 17.14% was achieved. 1 Introduction Backchannel (BC) is a short and quick reac- tion, such as "uh-huh" and "yes", of a listener to speaker’s utterances in a conversation (Yngve, 1970). Timely and appropriate use of BC in a con- versation is important because BC has various func- tional categories (Cutrone, 2010); and each cate- gory has different roles and influences on enriching a conversation (Heinz, 2003; Cohn et al., 2019; Lee et al., 2020). It is important to understand the speaker utterances for the proper use of BC. In this paper, we introduce a method to enhance the uti- lization of lexical information in utterances for BC category prediction. Utterances can be expressed by the acoustic and lexical information in spoken dialogue. In early BC prediction studies, there were several cases in which only the acoustic features of utterance were used (Ward and Tsukahara, 2000; Fujie et al., 2005; Poppe et al., 2010). Recently, the additional use of lexical information has improved the performance of the model (Ruede et al., 2017). However, the use of such lexical information was rather simple, including part-of-speech of the last word of utter- ance (Kawahara et al., 2016) or embedding features at the word level, such as Word2Vec. Ortega et al. (2020) used Word2Vec and reported the state-of- the-art performance level of about 58% accuracy on three types of BC categories, i.e. continuer, as- sessment, and No BC. Adiba et al. (2021a,b) also selected an approach similar to that of Ortega et al. (2020). However, they focused on addressing the la- tency of BC responses, not on utilization of lexical information. In this study, we introduce a BC Prediction Model using Multi-Task Learning (BPM_MT), which learns the main task of …

Content truncated to 500 words