Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddings

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7272 - 7282 December 7-11, 2022 ©2022 Association for Computational Linguistics Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddings Che Liu, Rui Wang, Junfeng Jiang, Yongbin Li∗, Fei Huang DAMO Academy, Alibaba Group {liuche.lc,wr224079,jiangjunfeng.jjf,shuide.lyb,f.huang}@alibaba-inc.com Abstract In this paper, we introduce the task of learning unsupervised dialogue embeddings. Trivial approaches such as combining pre-trained word or sentence embeddings and encoding through pre-trained language models (PLMs) have been shown to be feasible for this task. However, these approaches typically ignore the conversational interactions between interlocutors, resulting in poor performance. To address this issue, we proposed a self- guided contrastive learning approach named dial2vec. Dial2vec considers a dialogue as an information exchange process. It captures the conversational interaction patterns between interlocutors and leverages them to guide the learning of the embeddings corresponding to each interlocutor. The dialogue embed- ding is obtained by an aggregation of the embeddings from all interlocutors. To verify our approach, we establish a comprehensive benchmark consisting of six widely-used dialogue datasets. We consider three evalua- tion tasks: domain categorization, semantic relatedness, and dialogue retrieval. Dial2vec achieves on average 8.7, 9.0, and 13.8 points absolute improvements in terms of purity, Spearman’s correlation, and mean average precision (MAP) over the strongest baseline on the three tasks respectively. Further analysis shows that dial2vec obtains infor- mative and discriminative embeddings for both interlocutors under the guidance of the conversational interactions and achieves the best performance when aggregating them through the interlocutor-level pooling strategy. All codes and data are publicly available at https://github.com/AlibabaResearch/DAMO- ConvAI/tree/main/dial2vec. 1 Introduction Dialogue embedding, as a critical prerequisite of semantically understanding a dialogue, has been ∗ Corresponding author. a central issue in dialogue-related research such as dialogue clustering (Shi et al., 2018; Lv et al., 2021), conversational sentiment analysis (Wang et al., 2020; Lv et al., 2021), context-dependent text-to-SQL (Hui et al., 2021; Wang et al., 2022), and dialogue summarization (Liu et al., 2019b; Liu and Chen, 2021). Trivial unsupervised approaches generally encode dialogues by combining their pre- trained word or sentence embeddings (Pennington et al., 2014; Reimers and Gurevych, 2019) or us- ing PLMs (Wu et al., 2020a; Bao et al., 2020; He et al., 2022a,b,c). However, such methods are not specifically designed for dialogues and thus fail to adequately capture the key conversational infor- mation. In this paper, we formally introduce the task of learning unsupervised dialogue embeddings, which aims to learn dialogue embeddings that can well reflect conversational semantics without any additional manual annotations. Previous studies have extensively demonstrated the importance of encoding token-level interac- tions for learning semantic textual embeddings. However, for dialogue embedding, encoding interlocutor-level interactions is also essential but is overlooked in trivial approaches. Figure 1 shows I want to go out and do something. Perhaps you want to go to see some live music or to a sports event? Do you know what city you want to go? Maybe a concert? I love jazz. If possible, I’d like a convert …

Content truncated to 500 words