Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input

Proceedings of NAACL-HLT 2018, pages 263–271 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input Youmna Farag Helen Yannakoudakis Ted Briscoe Department of Computer Science and Technology The ALTA Institute University of Cambridge United Kingdom {youmna.farag,helen.yannakoudakis,ted.briscoe}@cl.cam.ac.uk Abstract We demonstrate that current state-of-the- art approaches to Automated Essay Scoring (AES) are not well-suited to capturing adver- sarially crafted input of grammatical but inco- herent sequences of sentences. We develop a neural model of local coherence that can ef- fectively learn connectedness features between sentences, and propose a framework for in- tegrating and jointly training the local coher- ence model with a state-of-the-art AES model. We evaluate our approach against a number of baselines and experimentally demonstrate its effectiveness on both the AES task and the task of flagging adversarial input, further contribut- ing to the development of an approach that strengthens the validity of neural essay scor- ing models. 1 Introduction Automated Essay Scoring (AES) focuses on auto- matically analyzing the quality of writing and as- signing a score to the text. Typically, AES models exploit a wide range of manually-tuned shallow and deep linguistic features (Shermis and Ham- mer, 2012; Burstein et al., 2003; Rudner et al., 2006; Williamson et al., 2012; Andersen et al., 2013). Recent advances in deep learning have shown that neural approaches to AES achieve state-of-the-art results (Alikaniotis et al., 2016; Taghipour and Ng, 2016) with the additional ad- vantage of utilizing features that are automatically learned from the data. In order to facilitate in- terpretability of neural models, a number of visu- alization techniques have been proposed to iden- tify textual (superficial) features that contribute to model performance (Alikaniotis et al., 2016). To the best of our knowledge, however, no prior work has investigated the robustness of neural AES systems to adversarially crafted input that is designed to trick the model into assigning desired missclassifications; for instance, a high score to a low quality text. Examining and addressing such validity issues is critical and imperative for AES deployment. Previous work has primarily focused on assessing the robustness of “standard” machine learning approaches that rely on manual feature engineering; for example, Powers et al. (2002); Yannakoudakis et al. (2011) have shown that such AES systems, unless explicitly designed to handle adversarial input, can be susceptible to subversion by writers who understand something of the sys- tems’ workings and can exploit this to maximize their score. In this paper, we make the following contribu- tions: i. We examine the robustness of state-of-the-art neural AES models to adversarially crafted input,1 and specifically focus on input related to local coherence; that is, grammatical but incoherent sequences of sentences.2 In addi- tion to the superiority in performance of neu- ral approaches against “standard” machine learning models (Alikaniotis et al., 2016; Taghipour and Ng, 2016), such a setup allows us to investigate their potential superiority / capacity in handling adversarial input with- out being explicitly designed …

Content truncated to 500 words