Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 153–162, Vancouver, Canada, August 3 - August 4, 2017. c⃝2017 Association for Computational Linguistics Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring Fei Dong and Yue Zhang∗and Jie Yang Singapore University of Technology and Design {fei dong, jie yang}@mymail.sutd.edu.sg yue zhang@sutd.edu.sg Abstract Neural network models have recently been applied to the task of automatic essay scor- ing, giving promising results. Existing work used recurrent neural networks and convolutional neural networks to model input essays, giving grades based on a sin- gle vector representation of the essay. On the other hand, the relative advantages of RNNs and CNNs have not been compared. In addition, different parts of the essay can contribute differently for scoring, which is not captured by existing models. We ad- dress these issues by building a hierarchi- cal sentence-document model to represent essays, using the attention mechanism to automatically decide the relative weights of words and sentences. Results show that our model outperforms the previous state- of-the-art methods, demonstrating the ef- fectiveness of the attention mechanism. 1 Introduction Automatic essay scoring (AES) is the task of au- tomatically assigning grades to student essays. It can be highly challenging, requiring not only knowledge on spelling and grammars, but also on semantics, discourse and pragmatics. Tradi- tional models use sparse features such as bag- of-words, part-of-speech tags, grammar complex- ity measures, word error rates and essay lengths, which can suffer from the drawbacks of time- consuming feature engineering and data sparsity. Recently, neural network models have been used for AES (Alikaniotis et al., 2016; Dong and Zhang, 2016; Taghipour and Ng, 2016), giving better results compared to statistical models with handcrafted features. In particular, distributed word representations are used for the input, and ∗ Corresponding author. a neural network model is employed to combine word information, resulting in a single dense vec- tor form of the whole essay. A score is given based on a non-linear neural layer on the representa- tion. Without handcrafted features, neural network models have been shown to be more robust than statistical models across different domains (Dong and Zhang, 2016). Both recurrent neural networks (Williams and Zipser, 1989; Mikolov et al., 2010) and convolu- tional neural networks (LeCun et al., 1998; Kim, 2014) have been used for modelling input es- says. In particular, Alikaniotis et al. (2016) and Taghipour and Ng (2016) use a single-layer LSTM (Hochreiter and Schmidhuber, 1997) over the word sequence to model the essay, and Dong and Zhang (2016) use a two-level hierarchical CNN structure to model sentences and documents separately. It has been commonly understood that CNNs can capture local ngram information effec- tively, while LSTMs are strong in modelling long history. No previous work has compared the ef- fectiveness of LSTMs and CNNs under the same settings for AES. To better understand the con- trast, we adopt the two-layer structure of Dong and Zhang (2016), comparing CNNs and LSTMs for modelling sentences and documents. Not …

Content truncated to 500 words