Towards Debate Automation: a Recurrent Model for Predicting Debate Winners

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2465–2475 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Towards Debate Automation: a Recurrent Model for Predicting Debate Winners Peter Potash and Anna Rumshisky Department of Computer Science University of Massachusetts Lowell {ppotash,arum}@cs.uml.edu Abstract In this paper we introduce a practical first step towards the creation of an auto- mated debate agent: a state-of-the-art re- current predictive model for predicting de- bate winners. By having an accurate pre- dictive model, we are able to objectively rate the quality of a statement made at a specific turn in a debate. The model is based on a recurrent neural network ar- chitecture with attention, which allows the model to effectively account for the entire debate when making its prediction. Our model achieves state-of-the-art accuracy on a dataset of debate transcripts anno- tated with audience favorability of the de- bate teams. Finally, we discuss how future work can leverage our proposed model for the creation of an automated debate agent. We accomplish this by determining the model input that will maximize audience favorability toward a given side of a debate at an arbitrary turn. 1 Introduction Conversational agents are a well-researched area of natural language generation (Pilato et al., 2007; Bigham et al., 2008; Augello et al., 2008; Agostaro et al., 2005; Bessho et al., 2012). Else- where in the field of natural language generation, there is work that seeks to generate persuasive text (Carenini and Moore, 2006; Reiter et al., 2003; Rosenfeld and Kraus, 2016), which is a logical first step towards creating an automated debate agent. One major deficiency of existing work in this area is its assessment of how convincing (or compelling) a piece of text is; the approaches use theory-driven models of persuasion, rather than being empirically motivated. Furthermore, none of these works provide a model that can optimize persuasiveness at an arbitrary point in a conversa- tion. One of the main reasons for a lack of empirically-driven persuasive generation systems is the absence of labeled data. In order to allevi- ate this problem (though not directly for the sake of producing an automated debate agent), Zhang et al. (2016) have introduced a dataset of debate transcripts from the “Intelligence Squared” (IQ2)1 debates. In these debates, two teams are present, arguing either for or against a given topic. For each debate, an audience poll is conducted both prior to and after the debate. Whichever team has the largest gain in audience support between the pre/post debate polls is the winner. This is a natu- ral way to account for the fact that some sides of a debate may be harder to argue than others, and that audience members may be initially biased given a debate topic. Because of the sequential nature of debating, a Recurrent Neural Network (RNN) is an attractive choice for modeling the problem. Rather than just using the final hidded state for prediction, which likely has lost information from early in …

Content truncated to 500 words