Stance Classification of Ideological Debates: Data, Models, Features, and Constraints

International Joint Conference on Natural Language Processing, pages 1348–1356, Nagoya, Japan, 14-18 October 2013. Stance Classification of Ideological Debates: Data, Models, Features, and Constraints Kazi Saidul Hasan and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {saidul,vince}@hlt.utdallas.edu Abstract Determining the stance expressed in a post written for a two-sided debate in an on- line debate forum is a relatively new and challenging problem in opinion mining. We seek to gain a better understanding of how to improve machine learning ap- proaches to stance classification of ideo- logical debates, specifically by examining how the performance of a learning-based stance classification system varies with the amount and quality of the training data, the complexity of the underlying model, the richness of the feature set, as well as the application of extra-linguistic constraints. 1 Introduction Determining the stance expressed in a post written for a two-sided debate in an online debate forum is a relatively new task in opinion mining. Given a post written for a two-sided topic discussed in an online debate forum (e.g., “Should abortion be banned?”), the goal of debate stance classifica- tion is to determine which of the two sides (i.e., for and against) its author is taking. Previous approaches to debate stance classifica- tion have focused on three debate settings, namely congressional floor debates (e.g., Thomas et al. (2006), Bansal et al. (2008), Balahur et al. (2009), Yessenalina et al. (2010), Burfoot et al. (2011)), company-internal discussions (e.g., Agrawal et al. (2003), Murakami and Raymond (2010)), and on- line social, political, and ideological debates in public forums (e.g., Somasundaran and Wiebe (2010), Wang and Ros´e (2010), Anand et al. (2011), Biran and Rambow (2011), Hasan and Ng (2012)). As Walker et al. (2012) point out, debates in public forums differ from congressional debates and company-internal discussions in terms of lan- guage use. Specifically, online debaters use color- ful and emotional language to express their points, which may involve sarcasm, insults, and question- ing another debater’s assumptions and evidence. These properties could make stance classification of online debates more challenging than that of the other two types of debates. Our goal in this paper is to gain a better under- standing of how to improve machine learning ap- proaches to stance classification of online debates by examining the following questions, which can be broadly categorized along four dimensions: Data. Can we improve the performance of a stance classification system simply by increasing the number of stance-annotated debate posts avail- able for training? Note, however, that the num- ber of stance-annotated posts that can be down- loaded from debate forums for a given debate do- main (e.g., Abortion) is fairly limited. A natural question is: given a debate domain, can we iden- tify from different data sources a large number of documents where authors express viewpoints rele- vant to the domain (e.g., blog posts, news articles) and then stance-label them heuristically, with the goal of employing these noisily labeled documents as additional data …

Content truncated to 500 words