Building a TOCFL Learner Corpus for Chinese Grammatical Error Diagnosis

Building a TOCFL Learner Corpus for Chinese Grammatical Error Diagnosis Lung-Hao Lee1,2, Yuen-Hsien Tseng2, Li-Ping Chang3 1MOST Joint Research Center for AI Technology and All Vista Healthcare, National Taiwan University 2Graduate Institute of Library and Information Studies, National Taiwan Normal University 3Mandarin Training Center, National Taiwan Normal University 1No. 1, Sec. 4, Roosevelt Rd., Taipei 10617, Taiwan 2,3No. 162, Sec. 1, Heping East Rd., Taipei 10610, Taiwan {lhlee, samtseng, lchang}@ntnu.edu.tw Abstract This study describes the construction of a TOCFL learner corpus and its usage for Chinese grammatical error diagnosis. We collected essays from the Test Of Chinese as a Foreign Language (TOCFL) and annotated grammatical errors using hierarchical tagging sets. Two kinds of error classifications were used simultaneously to tag grammatical errors. The first capital letter of each error tags denotes the coarse-grained surface differences, while the subsequent lowercase letters denote the fine-grained linguistic categories. A total of 33,835 grammatical errors in 2,837 essays and their corresponding corrections were manually annotated. We then used the Standard Generalized Markup Language to format learner texts and annotations along with learners’ accompanying metadata. Parts of the TOCFL learner corpus have been provided for shared tasks on Chinese grammatical error diagnosis. We also investigated systems participating in the shared tasks to better understand current achievements and challenges. The datasets are publicly available to facilitate further research. To our best knowledge, this is the first annotated learner corpus of traditional Chinese, and the entire learner corpus will be publicly released. Keywords: computer-assisted language learning, second language acquisition, grammatical error diagnosis, interlanguage analysis 1. Introduction Annotating learners’ inappropriate usage of written language is an important task for learner corpus research (Díaz-Negrillo and Fernández-Domínguez, 2006; Tono, 2003). From a linguistic perspective, annotated learner corpora are valuable resources for research in second language acquisition (Swanson and Charniak, 2013), foreign language teaching (Wang and Seneff, 2007), and contrastive interlanguage analysis (Granger, 2015). In engineering, such language resources can be used to develop natural language processing techniques for educational applications, such as automatic essay scoring (Yannakoudakis et al., 2011), assessment report generation (Sakaguchi et al., 2013), and native language identification (Malmasi and Dras, 2015). Automated grammatical error detection and correction are important research directions and a number of competitions have been organized to encourage innovation (Leacock et al., 2014). For example, Helping Our Own (HOO) is a series of shared tasks for correcting errors in non-native English texts (Dale and Kilgarriff, 2011, Dale et al., 2012). The CoNLL 2013/2014 shared tasks aimed to correct grammatical errors for learners of English as a foreign language (Ng et al., 2013; 2014). The NLPTEA workshops had hosted a series of shared tasks for Chinese grammatical error diagnosis (Yu et al., 2014; Lee et al., 2015b; Lee et al., 2016b). Recently, the IJCNLP 2017 shared task 1 has focused on Chinese grammatical error diagnosis (Rao et al., 2017). All these competitions require annotated learner corpora for system development and evaluation. To our best knowledge, only two previous studies have manually annotated learner corpora for …

Content truncated to 500 words