Lightly-Supervised Modeling of Argument Persuasiveness

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 594–604, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Lightly-Supervised Modeling of Argument Persuasiveness Isaac Persing and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {persingq,vince}@hlt.utdallas.edu Abstract We propose the first lightly-supervised ap- proach to scoring an argument’s persua- siveness. Key to our approach is the novel hypothesis that lightly-supervised persua- siveness scoring is possible by explicitly modeling the major errors that negatively impact persuasiveness. In an evaluation on a new annotated corpus of online debate arguments, our approach rivals its fully- supervised counterparts in performance by four scoring metrics when using only 10% of the available training instances. 1 Introduction Argumentation mining is a relatively new and ac- tive area of research in the natural language pro- cessing (NLP) community, focusing on extract- ing argument components (e.g., claims, premises) and determining the relationships (e.g., support, attack) between them. Recently, researchers have begun work on modeling an intriguing linguistic phenomenon, the persuasiveness of arguments. In this paper, we examine argument persuasive- ness in the context of an under-investigated task in argument mining, argument persuasiveness scor- ing. Given a text consisting of an argument writ- ten for a particular topic, the goal of argument persuasiveness scoring is to assign a score to the text that indicates how persuasive the argument is. An argument persuasiveness scoring system can be used in a variety of situations. In an online de- bate, for instance, an author’s primary goal is to convince others of the argument expressed in her comment(s). Similarly, in persuasive essay writ- ing, an author should establish convincing argu- ments. In both situations, a persuasiveness scor- ing system could provide useful feedback to these authors on how persuasive their arguments are. Being a discourse-level task, argument persua- siveness scoring is potentially more challenging than many NLP tasks. Oftentimes, argument per- suasiveness can only be determined by under- standing the discourse, not by the presence or ab- sence of lexical cues. As an example, consider the debate argument shown in Table 1, which is com- posed of the author’s assertion and her justifica- tion of the assertion written in response to a debate motion. It is fairly easy for a human to determine that this argument should be assigned a low per- suasiveness score because the argument could be more clear. However, the same is not true for a ma- chine, primarily because it is not possible to deter- mine the persuasiveness of this argument merely by considering the words or phrases appearing in it. Given the difficulty of the task, it is conceiv- able that unsupervised argument persuasiveness scoring is very challenging. Nevertheless, a so- lution to unsupervised argument persuasiveness scoring is of practical significance. This is be- cause of the high cost associated with manually creating persuasiveness-annotated data needed to train classifiers in a supervised manner. This con- trasts with tasks such as polarity classification and stance classification. In …

Content truncated to 500 words