Using Context to Predict the Purpose of Argumentative Writing Revisions

Proceedings of NAACL-HLT 2016, pages 1424–1430, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Using Context to Predict the Purpose of Argumentative Writing Revisions Fan Zhang Department of Computer Science University of Pittsburgh Pittsburgh, PA, 15260 zhangfan@cs.pitt.edu Diane Litman Department of Computer Science and LRDC University of Pittsburgh Pittsburgh, PA, 15260 litman@cs.pitt.edu Abstract While there is increasing interest in automat- ically recognizing the argumentative structure of a text, recognizing the argumentative pur- pose of revisions to such texts has been less explored. Furthermore, existing revision clas- sification approaches typically ignore contex- tual information. We propose two approaches for utilizing contextual information when pre- dicting argumentative revision purposes: de- veloping contextual features for use in the classification paradigm of prior work, and transforming the classification problem to a sequence labeling task. Experimental results using two corpora of student essays demon- strate the utility of contextual information for predicting argumentative revision purposes. 1 Introduction Incorporating natural language processing into sys- tems that provide writing assistance beyond gram- mar is an area of increasing research and commer- cial interest (e.g., (Writelab, 2015; Roscoe et al., 2015)). As one example, the automatic recognition of the purpose of each of an author’s revisions allows writing assistance systems to provide better rewrit- ing suggestions. In this paper, we propose context- based methods to improve the automatic identifica- tion of revision purposes in student argumentative writing. Argumentation plays an important role in analyzing many types of writing such as persuasive essays (Stab et al., 2014), scientific papers (Teufel, 2000) and law documents (Palau and Moens, 2009). In student papers, identifying revision purposes with respect to argument structure has been used to pre- dict the grade improvement in the paper after revi- sion (Zhang and Litman, 2015). Existing works on the analysis of writing revi- sions (Adler et al., 2011; Bronner and Monz, 2012; Daxenberger and Gurevych, 2013; Zhang and Lit- man, 2015) typically compare two versions of a text to extract revisions, then classify the purpose of each revision in isolation. That is, while limited con- textual features such as revision location have been utilized in prior work, such features are computed from the revision being classified but typically not its neighbors. In addition, ordinary classifiers rather than structured prediction models are typically used. To increase the role of context during prediction, in this paper we 1) introduce new contextual features (e.g., the impact of a revision on local text cohesion), and 2) transform revision purpose classification to a sequential labeling task to capture dependencies among revisions (as in Table 1). An experimental evaluation demonstrates the utility of our approach. 2 Related Work There are multiple works on the classification of revisions (Adler et al., 2011; Javanmardi et al., 2011; Bronner and Monz, 2012; Daxenberger and Gurevych, 2013; Zhang and Litman, 2015). While different classification tasks were explored, similar approaches were taken by extracting features (lo- cation, text, meta-data, language) from the revised text to train a classification model (SVM, Random Forest, etc.) on …

Content truncated to 500 words