Automated Chinese Essay Scoring from Multiple Traits

Proceedings of the 29th International Conference on Computational Linguistics, pages 3007–3016 October 12–17, 2022. 3007 Automated Chinese Essay Scoring from Multiple Traits Yaqiong He, Feng Jiang, Xiaomin Chu and Peifeng Li∗ School of Computer Science and Technology, Soochow University, Suzhou, China {20204227070,fjiang}@stu.suda.edu.cn {xmchu,pfli}@suda.edu.cn Abstract Automatic Essay Scoring (AES) is the task of using the computer to evaluate the quality of essays automatically. Current research on AES focuses on scoring the overall quality or single trait of prompt-specific essays. However, users expect to obtain not only the overall score but also the instant feedback from different traits to help improve their writing. Therefore, we first annotate a multi-trait dataset ACEA in- cluding 1220 argumentative essays from four traits, i.e., the essay organization, topic, logic, and language. And then we design a Hierarchi- cal Multi-task Trait Scorer (HMTS) to evaluate the quality of writing by modeling these four traits. Moreover, we propose an inter-sequence attention mechanism to enhance information in- teraction between different tasks and design the trait-specific features for various tasks in AES. The experimental results on ACEA show that our HMTS can effectively score essays from multiple traits, outperforming several strong baselines. 1 Introduction Automatic Essay Scoring (AES) is a task of using the computer to evaluate the quality of students’ es- says. An efficient AES system can bring many ben- efits to the field of education, including avoiding the influence of subjective factors on essay scoring, greatly reducing teachers’ workload, and providing instant feedback for students’ written essays. Early work treated AES as a classification (Larkey, 1998; Rudner and Liang, 2002), regres- sion (Attali and Burstein, 2005; Phandi et al., 2015), or ranking problem (Yannakoudakis et al., 2011; Chen and He, 2013), addressing AES by super- vised learning. Recently, with the progress of deep learning, the end-to-end models based on neural networks (e.g., LSTM-based models or the com- bination of LSTMs and CNNs) have achieved im- pressive results (Alikaniotis et al., 2016; Taghipour ∗Corresponding author and Ng, 2016; Dong and Zhang, 2016; Dong et al., 2017; Tay et al., 2018). In consideration for more feedback in AES sys- tems, some studies began to focus on scoring spe- cific traits of essays, such as word choice, content (Page, 1966), organization (Persing et al., 2010; Song et al., 2020a), thesis clarity (Persing and Ng, 2013), prompt adherence (Persing and Ng, 2014), argument persistent (Ke et al., 2018), style (Page, 1966), etc. Most of the above studies focused on scoring prompt-specific essays. As previous re- searchers had pointed out (Jin et al., 2018; Ridley et al., 2020), it was difficult and expensive to access ample target-prompt essays in the real-world AES system. Therefore, some researchers on English AES had began to explore cross-prompt AES (Rid- ley et al., 2020; Phandi et al., 2015; Jin et al., 2018). Their methods utilized non-target-prompt essays to train models and score target-prompt essays. In reality, people usually judge the essay from different perspectives to give the final overall holis- tic score. Apart from (Ridley et al., …

Content truncated to 500 words