Automated Essay Scoring via Pairwise Contrastive Regression

Proceedings of the 29th International Conference on Computational Linguistics, pages 2724–2733 October 12–17, 2022. 2724 Automated Essay Scoring via Pairwise Contrastive Regression Jiayi Xie∗, Kaiwei Cai∗, Li Kong, Junsheng Zhou†, Weiguang Qu NLP Lab, Department of Computer and Electronic Information Nanjing Normal University, China { tammyjiayixie, ckwcccwkc }@gmail.com kongli@nnu.edu.cn, { zhoujs, wgqu }@njnu.edu.cn Abstract Automated essay scoring (AES) involves the prediction of a score relating to the writing qual- ity of an essay. Most existing works in AES utilize regression objectives or ranking objec- tives respectively. However, the two types of methods are highly complementary. To this end, in this paper we take inspiration from contrastive learning and propose a novel uni- fied Neural Pairwise Contrastive Regression (NPCR) model in which both objectives are op- timized simultaneously as a single loss. Specif- ically, we first design a neural pairwise ranking model to guarantee the global ranking order in a large list of essays, and then we further extend this pairwise ranking model to predict the rela- tive scores between an input essay and several reference essays. Additionally, a multi-sample voting strategy is employed for inference. We use Quadratic Weighted Kappa to evaluate our model on the public Automated Student Assess- ment Prize (ASAP) dataset, and the experimen- tal results demonstrate that NPCR outperforms previous methods by a large margin, achieving the state-of-the-art average performance for the AES task1. 1 Introduction Automated Essay Scoring (AES) is to evaluate the quality of essays and score automatically by using computer technologies. Notably, reasonable grad- ing can solve problems that consume much time and require a lot of human effort. What’s more, providing feedback to learners can promote self improvement. It is one of the most important appli- cations of natural language processing (NLP) and is widely required in the educational field. Most existing methods typically recast AES as a regression task, where the goal is to predict the * Equal contribution. † Corresponding author. 1The source code is available at https://github. com/CarryCKW/AES-NPCR. score of an essay (Taghipour and Ng, 2016; Dong and Zhang, 2016; Dong et al., 2017; Tay et al., 2018). Although some promising results have been achieved, these regression-based models cannot ex- ploit the labelling information in the training data efficiently and directly. Besides, another line of research treats AES as a preference ranking prob- lem with learning-to-rank methods. Yannakoudakis et al. (2011) first proposed to rank the pair of doc- uments by extracting features; later, Chen and He (2013) transformed this task into a listwise ranking problem. Cummins et al. (2016) also performed transfer learning to rank two essays that are con- strained to be from the same prompt. However, most existing works in AES utilize regression objectives or ranking objectives respec- tively. As a matter of fact, the two types of methods are highly complementary. On the one hand, only using regression models for AES cannot explic- itly model score relationships between essays in the training data. On the other hand, only using ranking-based models could not guarantee accu- …

Content truncated to 500 words