Chat Disentanglement: Identifying Semantic Reply Relationships with Random Forests and Recurrent Neural Networks

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 615–623, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Chat Disentanglement: Identifying Semantic Reply Relationships with Random Forests and Recurrent Neural Networks Shikib Mehri Department of Computer Science University of British Columbia Vancouver, Canada mehrishikib@gmail.com Giuseppe Carenini Department of Computer Science University of British Columbia Vancouver, Canada carenini@cs.ubc.ca Abstract Thread disentanglement is a precursor to any high-level analysis of multipartici- pant chats. Existing research approaches the problem by calculating the likelihood of two messages belonging in the same thread. Our approach leverages a newly annotated dataset to identify reply rela- tionships. Furthermore, we explore the us- age of an RNN, along with large quanti- ties of unlabeled data, to learn semantic relationships between messages. Our pro- posed pipeline, which utilizes a reply clas- sifier and an RNN to generate a set of dis- entangled threads, is novel and performs well against previous work. 1 Introduction The problem of thread disentanglement is a pre- cursor to high-level analysis of multiparticipant chats (Carenini et al., 2011). A typical chat con- sists of multiple simultaneous and distinct con- versations, with Elsner and Charniak (2010) ob- serving an average of 2.75 simultaneous threads of dialogue. Since a conversation does not neces- sarily entail a contiguous sequence of messages, the interwoven threads must be identified and seg- mented prior to any high-level analysis of the chat. To further illustrate the need for thread disen- tanglement, consider the chat log in Figure 1. It should be clear to a reader that there are two in- dependent threads of dialogue occurring within this sequence of messages, the first between John, Jack and Brian and the second between Jenny and Katie. Humans are adept at mentally disentan- gling conversations and even go as far as to ad- just their behavior in order to ease the process of disentanglement which O’Neill and Martin (2003) observed in the form of name mentioning. John: i need a new tv show to watch Jack: psych/house of cards/breaking bad sound like things you might enjoy Brian: oh I should probably renew my Netflix Katie: I forgot my laprop at home D: Jenny: Katie, that sucks... Jenny: Are you going to go back home to get it? Katie: laptop* Brian: try Black Mirror Figure 1. An example of a multiparticipant chat with two threads of dialogue. Our work is novel because it approaches the problem of thread disentanglement by attempt- ing to predict immediate reply relationships be- tween messages. The potential benefits of this idea were discussed by Elsner and Charniak (2010) and Uthus and Aha (2013), but the idea has not yet been explored. Furthermore, Elsner and Charniak (2010) suggest that this approach ”might yield more reliable annotations”. Additionally, we explore the usage of unlabeled data for the purpose of identifying semantic rela- tionships between messages. Previous attempts at semantic modeling by Elsner and Charniak (2010) and Adams and Martel (2010) have not been very effective, however recent …

Content truncated to 500 words