Detecting Disagreement in Conversations using Pseudo-Monologic Rhetorical Structure

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1169–1180, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Detecting Disagreement in Conversations using Pseudo-Monologic Rhetorical Structure Kelsey Allen Giuseppe Carenini Raymond T. Ng Department of Computer Science, University of British Columbia Vancouver, BC, V6T 1Z4, Canada {kelseyra, carenini, rng}@cs.ubc.ca Abstract Casual online forums such as Reddit, Slashdot and Digg, are continuing to in- crease in popularity as a means of com- munication. Detecting disagreement in this domain is a considerable challenge. Many topics are unique to the conversa- tion on the forum, and the appearance of disagreement may be much more sub- tle than on political blogs or social me- dia sites such as twitter. In this analy- sis we present a crowd-sourced annotated corpus for topic level disagreement detec- tion in Slashdot, showing that disagree- ment detection in this domain is difficult even for humans. We then proceed to show that a new set of features determined from the rhetorical structure of the con- versation significantly improves the per- formance on disagreement detection over a baseline consisting of unigram/bigram features, discourse markers, structural fea- tures and meta-post features. 1 Introduction How does disagreement arise in conversation? Be- ing able to detect agreement and disagreement has a range of applications. For an online educator, dissent over a newly introduced topic may alert the teacher to fundamental misconceptions about the material. For a business, understanding dis- putes over features of a product may be helpful in future design iterations. By better understanding how debate arises and propagates in a conversa- tion, we may also gain insight into how authors’ opinions on a topic can be influenced over time. The long term goal of our research is to lay the foundations for understanding argumentative structure in conversations, which could be applied to NLP tasks such as summarization, information retrieval, and text visualization. Argumentative structure theory has been thoroughly studied in both psychology and rhetoric, with negation and discourse markers, as well as hedging and dis- preferred responses, being known to be indicative of argument (Horn, 1989; Brown and Levinson, 1987). As a starting point, in this paper we focus on the detection of disagreement in casual con- versations between users. This requires a gener- alized approach that can accurately identify dis- agreement in topics ranging from something as mundane as whether GPS stands for galactic po- sitioning system or global positioning system, to more ideological debates about distrust in science. Motivated by the widespread consensus in both computational and theoretical linguistics on the utility of discourse markers for signalling prag- matic functions such as disagreement and personal opinions (Webber and Prasad, 2008; Abbott et al., 2011; J. E. Fox-Tree, 2010), we introduce a new set of features based on the Discourse Tree (DT) of a conversational text. Discourse Trees were formalized by Mann and Thompson (1988) as part of their Rhetorical Structure Theory (RST) to represent the structure of discourse. Although this theory is for …

Content truncated to 500 words