Context-Aware Conversation Thread Detection in Multi-Party Chat

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6456–6461, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6456 Context-Aware Conversation Thread Detection in Multi-Party Chat Ming Tan ⇤† Dakuo Wang⇤‡ Yupeng Gao⇤‡ Haoyu Wang† Saloni Potdar† Xiaoxiao Guo‡ Shiyu Chang ‡ Mo Yu‡ †IBM Watson ‡IBM Research Abstract In multi-party chat, it is common for multiple conversations to occur concurrently, leading to intermingled conversation threads in chat logs. In this work, we propose a novel Context- Aware Thread Detection (CATD) model that automatically disentangles these conversation threads. We evaluate our model on three real- world datasets and demonstrate an overall im- provement in thread detection accuracy over state-of-the-art benchmarks. 1 Introduction In multi-party chat conversations, such as in Slack1, multiple topics are often discussed at the same time (Wang et al., 2019; Zhang et al., 2017). For example, in Table 1, Alice and Bob talk about work, while Bob and Chuck chat about lunch, and this results in intermingled messages. Automatic conversational thread detection could be used to disentangle and group the messages into their re- spective topic threads. The resulting thread infor- mation could in turn be used to improve response relevancy for conversational agents (Shamekhi et al., 2018) or improve chat summarization qual- ity (Zhang and Cranshaw, 2018). Unlike most of today’s Email and Forum sys- tems that use threaded structure by default. How- ever, the Instant Messaging systems (e.g., Slack) often require users to manually organize messages in threads. A recent study (Wang et al., 2019) found that users most likely do not manually create threads. On average, only 15.3 threads were cre- ated per Slack channel with 355 messages, when they discuss group projects. Prior work on conversation thread disentangle- ment is often based on pairwise message compar- ⇤ Equal contributions from the corresponding authors: mingtan@us.ibm.com, dakuowang@ibm.com, yupeng.gao@ibm.com. 1A instant messaging platform: https://slack.com Thread Message T1 Alice: Bob, please send me the proposal. T2 Bob: Where are we going for lunch? T1 Bob: Sent. T1 Alice: Thanks. T2 ? Chuck: How about the Thai place? Table 1: Messages and threads in a chat channel. Does Chuck’s message belong to T1, T2, or a new thread? ison. Some solutions use unsupervised cluster- ing methods with hand-engineered features (Wang and Oard, 2009; Shen et al., 2006), while oth- ers use supervised approaches with statistical (Du et al., 2017) or linguistic features (Wang et al., 2008; Wang and Ros´e, 2010; Elsner and Charniak, 2008, 2010, 2011; Mayfield et al., 2012). Recent work by (Jiang et al., 2018; Mayfield et al., 2012) adopt deep learning approaches to compute mes- sage pair similarity, using a combination of mes- sage content and simple contextual features (e.g., authorship and timestamps). However, linguistic theories (Biber and Con- rad, 2019) differentiate the following three con- cepts: register, genre and style, to describe the text varieties. Register refers to the linguistic features such as the choice of words …

Content truncated to 500 words