End-to-End Argumentation Mining in Student Essays

Proceedings of NAACL-HLT 2016, pages 1384–1394, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics End-to-End Argumentation Mining in Student Essays Isaac Persing and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {persingq,vince}@hlt.utdallas.edu Abstract Understanding the argumentative structure of a persuasive essay involves addressing two challenging tasks: identifying the components of the essay’s argument and identifying the relations that occur between them. We ex- amine the under-investigated task of end-to- end argument mining in persuasive student es- says, where we (1) present the first results on end-to-end argument mining in student es- says using a pipeline approach; (2) address error propagation inherent in the pipeline ap- proach by performing joint inference over the outputs of the tasks in an Integer Linear Pro- gramming (ILP) framework; and (3) propose a novel objective function that enables F-score to be maximized directly by an ILP solver. We evaluate our joint-inference approach with our novel objective function on a publicly- available corpus of 90 essays, where it yields an 18.5% relative error reduction in F-score over the pipeline system. 1 Introduction There has been a surge of interest in argumenta- tion mining in recent years. Argumentation mining typically involves addressing two subtasks: (1) ar- gument component identification (ACI), which con- sists of identifying the locations and types of the components that make up the arguments (i.e., Ma- jor Claims, Claims, and Premises), and (2) relation identification (RI), which involves identifying the type of relation that holds between two argument components (i.e., Support, Attack, None). As a first step towards mining arguments in persuasive essays, Stab and Gurevych (S&G) annotated a corpus of 90 student essays with argument components and their relations (Stab and Gurevych, 2014a). To illustrate, consider the following excerpt from one essay: From this point of view, I firmly believe that (1) we should attach more importance to co- operation during primary education. First of all, (2) through cooperation, children can learn about interpersonal skills which are significant in the future life of all students. (3) What we acquired from team work is not only how to achieve the same goal with others but more importantly, how to get along with others. In this example, premise (3) supports claim (2), which in turn supports major claim (1). Using their annotated corpus, S&G presented ini- tial results on simplified versions of the ACI and RI tasks (Stab and Gurevych, 2014b). Specifically, they applied their learned ACI classifier to classify only gold argument components (i.e., text spans corre- sponding to a Major Claim, Claim, or Premise in the gold standard) or sentences that contain no gold argument components (as non-argumentative). Sim- ilarly, they applied their learned RI classifier to clas- sify only the relation between two gold argument components. In other words, they simplified both tasks by avoiding the challenging task of identify- ing the locations of argument components. Conse- quently, their approach cannot be applied in a realis- tic setting where the input is …

Content truncated to 500 words