Length, Interchangeability, and External Knowledge: Observations from Predicting Argument Convincingness

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 342–351, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Length, Interchangeability, and External Knowledge: Observations from Predicting Argument Convincingness Peter Potash, Robin Bhattacharya, Anna Rumshisky Department of Computer Science University of Massachusetts Lowell {ppotash,rbhattac,arum}@cs.uml.edu Abstract In this work, we provide insight into three key aspects related to predicting argument convincingness. First, we explicitly dis- play the power that text length possesses for predicting convincingness in an unsu- pervised setting. Second, we show that a bag-of-words embedding model posts state-of-the-art on a dataset of arguments annotated for convincingness, outperform- ing an SVM with numerous hand-crafted features as well as recurrent neural net- work models that attempt to capture se- mantic composition. Finally, we as- sess the feasibility of integrating exter- nal knowledge when predicting convinc- ingness, as arguments are often more con- vincing when they contain abundant infor- mation and facts. We finish by analyzing the correlations between the various mod- els we propose. 1 Introduction Predicting argument convincingness has mostly been studied in relation to the overall quality of a persuasive essay (Attali and Burstein, 2004; Lan- dauer, 2003; Shermis et al., 2010), with a recent focus specifically on predicting argument strength (Persing and Ng, 2015; Wachsmuth et al., 2016). Zhang et al. (2016) have also attempted to pre- dict argument convincingness, in the form of pre- dicting debate winners. Unfortunately, these are very rare argumentative formats that are seldom encountered in everyday life. In practice, at least at the moment, we tend to digest a large quantity of our information from social media and engage in a tremendous amount of interpersonal communi- cation using it. Since, in social media, communi- cations are roughly a single paragraph, analyzing arguments in a persuasive essay or oxford-style debate is not applicable to our primary means of community engagement. Presenting an entire con- vincing argument within a single paragraph can be an invaluable skill in the modern world. This pa- per seeks to improve upon previous methodology for predicting argument convincingness. Prompt: Is it better to have a lousy father or to be fatherless? Stance: It is better to have a lousy father. Argument 1 Argument 2 It is better to have a lousy father because researchers at the McGill University have warned that agrowing up without father can per- manently change the structure of a child’s brain and make him/her more aggressive and angry. Having a lousy fa- ther is better because when a child does not have a father, it causes him/her to look for a father figure. Dur- ing such searches, a child may end up get- ting sexual harassed or being emotionally exploited to various degrees. Table 1: Example of an argument pair where Ar- gument 1 is more convincing. Habernal and Gurevych (2016b) have recently released a dataset of short, single-paragraph ar- guments annotated for convincingness, which we will refer to as UKPConvArg. For 16 issues, ar- guments with the same stance are …

Content truncated to 500 words