Zero-Shot Dialogue Disentanglement by Self-Supervised Entangled Response Selection

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4897–4902 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4897 Zero-Shot Dialogue Disentanglement by Self-Supervised Entangled Response Selection Ta-Chung Chi Language Technologies Institute Carnegie Mellon University tachungc@andrew.cmu.edu Alexander I. Rudnicky Language Technologies Institute Carnegie Mellon University air@cs.cmu.edu Abstract Dialogue disentanglement aims to group utter- ances in a long and multi-participant dialogue into threads. This is useful for discourse anal- ysis and downstream applications such as dia- logue response selection, where it can be the first step to construct a clean context/response set. Unfortunately, labeling all reply-to links takes quadratic effort w.r.t the number of ut- terances: an annotator must check all preced- ing utterances to identify the one to which the current utterance is a reply. In this paper, we are the first to propose a zero-shot dialogue disentanglement solution. Firstly, we train a model on a multi-participant response selec- tion dataset harvested from the web which is not annotated; we then apply the trained model to perform zero-shot dialogue disentan- glement. Without any labeled data, our model can achieve a cluster F1 score of 25. We also fine-tune the model using various amounts of labeled data. Experiments show that with only 10% of the data, we achieve nearly the same performance of using the full dataset1. 1 Introduction Multi-participant chat platforms such as Messen- ger and WhatsApp are common on the Internet. While being easy to communicate with others, mes- sages often flood into a single channel, entangling chat history which is poorly organized and difficult to structure. In contrast, Slack provides a thread- opening feature that allows users to manually orga- nize their discussions. It would be ideal if we could design an algorithm to automatically organize an entangled conversation into its constituent threads. This is referred to as the task of dialogue disentan- glement (Shen et al., 2006; Elsner and Charniak, 2008; Wang and Oard, 2009; Elsner and Charniak, 2011; Jiang et al., 2018; Kummerfeld et al., 2018; Zhu et al., 2020; Li et al., 2020; Yu and Joty, 2020). 1Code is released at https://github. Entangled Dialogue Shared Model Selection Loss Attention Loss New Entangled Dialogue Dialogue Disentanglement self-supervised training zero shot adaption Figure 1: This is the high-level flow of our proposed approach. Training data for the dialogue disentanglement task is difficult to acquire due to the need for manual annotation. Typically, the data is anno- tated in the reply-to links format, i.e. every utter- ance is linked to one preceding utterance. The effort is quadratic w.r.t the length of dialogue, partly explaining the sole existence of human- annotated large-scale dataset (Kummerfeld et al., 2018), which was constructed based on the Ubuntu IRC forum. To circumvent the need for expen- sive labeled data, we aim to train a self-supervised model first then use the model to perform zero-shot dialogue disentanglement. In other words, our goal is to find a task that can learn implicit reply-to links without labeled data. Entangled response selection (Gunasekara …

Content truncated to 500 words