A Corpus of Annotated Revisions for Studying Argumentative Writing

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1568–1578 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1144 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1568–1578 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1144 A Corpus of Annotated Revisions for Studying Argumentative Writing Fan Zhang Homa B. Hashemi Rebecca Hwa Diane Litman University of Pittsburgh Pittsburgh, PA, 15260 {zhangfan,hashemi,hwa,litman}@cs.pitt.edu Abstract This paper presents ArgRewrite, a corpus of between-draft revisions of argumenta- tive essays. Drafts are manually aligned at the sentence level, and the writer’s pur- pose for each revision is annotated with categories analogous to those used in argu- ment mining and discourse analysis. The corpus should enable advanced research in writing comparison and revision analysis, as demonstrated via our own studies of student revision behavior and of automatic revision purpose prediction. 1 Introduction Most writing-related natural language process- ing (NLP) research focuses on the analysis of single drafts. Examples include document-level quality assessment (Attali and Burstein, 2006; Burstein and Chodorow, 1999), discourse-level analysis and mining (Burstein et al., 2003; Falak- masir et al., 2014; Persing and Ng, 2016), and fine-grained error detection (Leacock et al., 2010; Grammarly, 2016). Less studied is the analysis of changes between drafts – a comparison of re- visions and the properties of the differences. Re- search on this topic can support applications invol- ing revision analysis (Zhang and Litman, 2015), paraphrase (Malakasiotis and Androutsopoulos, 2011) and correction detection (Swanson and Ya- mangil, 2012; Xue and Hwa, 2014). Although there are some corpora resources for NLP research on writing comparisons, most tend to be between individual sentences/phrases for tasks such as paraphrase comparison (Dolan and Brockett, 2005; Tan and Lee, 2014) or gram- mar error correction (Dahlmeier et al., 2013; Yannakoudakis et al., 2011). In terms of revi- sion analysis, the most relevant work analyzes Wikipedia revisions (Daxenberger and Gurevych, 2013; Bronner and Monz, 2012); however, the do- main of Wikipedia is so specialized that the prop- erties of Wikipedia revisions do not correspond well with other kinds of texts. This work presents the ArgRewrite corpus1 to facilitate revision analysis research for argumen- tative essays. The corpus consists of a collec- tion of three drafts of essays written by univer- sity students and employees; the drafts are man- ually aligned at the sentence level, then the pur- pose of each revision is manually coded using a revision schema closely related to argument min- ing/discourse analysis. Within the domain of argu- mentative essays, the corpus will be useful for sup- porting research in argumentative revision analy- sis and the application of argument mining tech- niques. The corpus may also be useful for re- search on paraphrase comparisons, grammar error correction, and computational stylistics (Popescu and Dinu, 2008; Flekova et al., 2016). In this pa- per, we present two example uses of our corpus: 1) rewriting behavior data analysis, and 2) automatic revision purpose classification. …

Content truncated to 500 words