Chinese Content Scoring: Open-Access Datasets and Features on Different Segmentation Levels

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 347–357 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 347 Chinese Content Scoring: Open-Access Data Sets and Features on Different Segmentation Levels Yuning Ding Andrea Horbach Haoshi Wang Language Technology Lab, University Duisburg-Essen (yuning.ding|andrea.horbach|torsten.zesch)@uni-due.de Xuefeng Song Torsten Zesch Abstract In this paper, we analyse the challenges of Chinese content scoring in comparison to English. As a review of prior work for Chi- nese content scoring shows a lack of open- access data in the field, we present two short-answer data sets for Chinese. The Chinese Educational Short Answers data set (CESA) contains 1800 student answers for five science-related questions. As a second data set, we collected ASAP-ZH with 942 answers by re-using three exist- ing prompts from the ASAP data set. We adapt a state-of-the-art content scoring system for Chinese and evaluate it in sev- eral settings on these data sets. Results show that features on lower segmentation levels such as character n-grams tend to have better performance than features on token level. 1 Introduction Short answer questions are a type of educa- tional assessment that requires respondents to give natural language answers in response to a question or some reading material (Rademak- ers et al., 2005). The applications used to automatically score such questions are usually thought of as content scoring systems, because content (and not linguistic form) is taken into consideration for automatic scoring (Ziai et al., 2012). While there is a large research body for English content scoring, there is less research for Chinese.1 The largest obstacle for more re- search on Chinese is the lack of publicly avail- able data sets of Chinese short answer ques- tions. 1In this work, we use the term ‘Chinese’ as abbrevi- ation for Mandarin Chinese, which includes simplified and traditional written Chinese. Cantonese, Wu, Min Nan and other dialects are not included. Working with Chinese poses substantially different challenges than work on English data. Unlike English, which uses spaces as natu- ral separators between words, segmentation of Chinese texts into tokens is challenging (Chen and Liu, 1992). Furthermore, there are more options on which level to segment Chinese text. Apart from tokenization and segmen- tation into characters, which are two options also available and often used for English, seg- mentation into components, radicals and even individual strokes are additionally possible for Chinese. Table 1 gives an example for the seg- mentation options in both languages. Ortho- graphic variance can be challenging in both languages, but behaves very differently. Non- word errors, which is the main source of ortho- graphic problems in English (Mitton, 1987), can by definition not happen in Chinese, due to the input modalities. Language Level Unigrams English word panda characters p, a, n, d, a Chinese word 熊猫 characters 熊，猫 components radicals 灬，犭 strokes ... Table 1: Comparison of segmentation possibilities in English and Chinese In the remainder of …

Content truncated to 500 words