Modeling Stance in Student Essays

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2174–2184, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Modeling Stance in Student Essays Isaac Persing and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {persingq,vince}@hlt.utdallas.edu Abstract Essay stance classification, the task of de- termining how much an essay’s author agrees with a given proposition, is an important yet under-investigated subtask in understanding an argumentative essay’s overall content. We introduce a new cor- pus of argumentative student essays an- notated with stance information and pro- pose a computational model for automati- cally predicting essay stance. In an evalu- ation on 826 essays, our approach signif- icantly outperforms four baselines, one of which relies on features previously devel- oped specifically for stance classification in student essays, yielding relative error reductions of at least 11.3% and 5.3%, in micro and macro F-score, respectively. 1 Introduction State-of-the-art automated essay scoring engines such as E-rater (Attali and Burstein, 2006) do not grade essay content, focusing instead on pro- viding diagnostic trait feedback on categories such as grammar, usage, mechanics, style and organization. Hence, persuasiveness and other content-dependent dimensions of argumentative essay quality are largely ignored in existing auto- mated essay scoring research. While full-fledged content-based essay scoring is still beyond the reach of state-of-the-art essay scoring engines, re- cent work has enabled us to move one step closer to this ambitious goal by analyzing essay content, attempting to determine the argumentative struc- ture of student essays (Stab and Gurevych, 2014) and the persuasiveness of the arguments made in these essays (Persing and Ng, 2015). Stance classification is an important first step in determining how persuasive an argumentative stu- dent essay is because persuasiveness depends on how well the author argues w.r.t. the stance she takes using the supporting evidence she provides. For instance, if her stance is Agree Somewhat, a persuasive argument would involve explaining what reservations she has about the given propo- sition. As another example, an argumentative es- say in which the author takes a neutral stance or the author presents evidence that does not support the stance she claims to take should receive a low persuasiveness score. Given the important role played by stance classification in determining an essay’s persua- siveness, our goal in this paper is to examine stance classification in argumentative student es- says. While there is a large body of work on stance classification1, stance classification in argumen- tative essays is largely under-investigated and is different from previous work in several respects. First, in automated essay grading, the majority of the essays to be assessed are written by students who are learners of English. Hence our stance classification task could be complicated by the au- thors’ lack of fluency in English. Second, essays are longer and more formally written than the text typically used in previous stance classification re- search (e.g., debate posts). In particular, a student essay writer typically expresses her stance on the essay’s …

Content truncated to 500 words