Enhancing Automatic Chinese Essay Scoring System from Figures-of-Speech

Enhancing Automatic Chinese Essay Scoring System from Figures-of-Speech Tao-Hsing Chang1,2, Chia-Hoang Lee2, and Yu-Ming Chang2 1 Research Center for Psychological and Educational Testing, National Taiwan Normal University, 10610 Taipei, Taiwan 2 Institute of Computer Science and Engineering, National Chiao Tung University, 30010 Hsinchu, Taiwan {thchang, chl, Porter}@cis.nctu.edu.tw Abstract. Chinese automated essay scoring (CAES) is a very important tool for many educational researches. However, none of the methods for retrieving features in English essays is applicable to Chinese writings because Chinese grammar parsers cannot produce reliable and useful syntactic features. CAES systems must explore other distinct features in Chinese writing. This paper proposes a method for retrieving two Chinese figures-of-speech from essays and to predict the scores of essays based on the two skills used in the essays. A scoring model based on ID3 decision tree algorithm will validate and show that the proposed method increases the performance of CAES effectively. Keywords: Automated essay scoring; Figures-of-speech; Chinese writing; ID3. 1 Introduction Automated essay scoring (AES) in Chinese is a very important tool for such educational researches as curriculum and instruction, psychometrics, and educational testing. The studies on above domains usually use large writing samples to analyze the relationship between the score and the writer of an essay. The task, requiring many certified raters or experts, is very costly and time consuming and therefore impedes the development of many important and interesting studies on these domains. It is evident and urgent to develop theories and techniques for machine to automatically grade essays. Many studies ([1], [2], [3], [4], and [5]) for AES in English have been proposed and successfully applied to business products because of breakthrough in the field of natural language processing and information retrieval [6]. These approaches mainly contain three phases: retrieving the features of essays, modeling the associations between the score and the features in essays, and constructing a model for predicting essay scores using machine-learning methods. However, none of the methods for retrieving features in English essays is applicable to Chinese writings. This is due to a loose and ambiguous definition of Chinese sentence. Furthermore, current Chinese grammar parsers cannot produce reliable and useful syntactic features due to its low precision rates. Hence, the technique for developing CAES systems must explore other distinct features in Chinese writings. Many studies for Chinese writing skills indicate that usage of figures-of-speech affects the score of an essay greatly [7]. It is interesting to note that there are three characteristics in figures-of-speech which can be used to facilitate CAES to better score Chinese essays. First, it is a writing skill beyond the beginners and requires extensive training. Essays containing figures-of-speech usually score better grades. Second, some subcategories of figures-of-speech are only dependent on both the arrangement of part-of-speech in sentences and the number of words. These subcategories are not related to the semantics of sentences and become feasible to extract. Third, most of figures-of-speech appear in intra- sentence or among few sentences. Hence, the features are not rare. This paper will focus …

Content truncated to 500 words