Towards Automatic Error Type Classification of Japanese Language Learners’ Writings

Towards Automatic Error Type Classification of Japanese Language Learners’ Writing Hiromi Oyama Graduate School of Information Science Nara Institute of Science and Technology 8916-5 Takayama, Ikoma, Nara, Japan hiromi-o@is.naist.jp Mamoru Komachi Graduate School of System Design Tokyo Metropolitan University 6-6 Asahigaoka, Hino, Tokyo, Japan komachi@tmu.ac.jp Yuji Matsumoto Graduate School of Information Science Nara Institute of Science and Technology 8916-5 Takayama, Ikoma, Nara, Japan matsu@is.naist.jp Abstract Learner corpora are receiving special atten- tion as an invaluable source of educational feedback and are expected to improve teach- ing materials and methodology. However, they include various types of incorrect sen- tences. Error type classification is an impor- tant task in learner corpora which enables clarifying for learners why a certain sen- tence is classified as incorrect in order to help learners not to repeat errors. To ad- dress this issue, we defined a set of error type criteria and conducted automatic clas- sification of errors into error types in the sentences from the NAIST Goyo Corpus and achieved an accuracy of 77.6%. We also tried inter-corpus evaluation of our system on the Lang-8 corpus of learner Japanese and achieved an accuracy of 42.3%. To know the accuracy, we also investigated the classification method by human judgement and compared the difference in classifica- tion between the machine and the human. 1 Introduction Automatic error detection is one area that has been widely studied. One of the challenges in this work is generalizing the great number of error patterns. Given that the different types of learners’ errors are too numerous to detect, some researchers have broken down the error detection task according to the types of errors, such as spelling errors, mass count noun errors and preposition errors. If the error type classification is made in advance, it will help the automatic error detection system more ac- curate. Classifying error types has other advantages. First, it will help resulting learner corpora useful in linguistic research. It can offer teachers with effective feedback on patterns of errors repeatedly made by students. Secondly, through classifica- tion of errors, learners are able to correct their own errors by comparing acceptable and unacceptable sentences. Learner corpora are useful for statistical analy- sis of learner output and provide positive and neg- ative examples that contribute to improving writ- ing skills. According to Ellis’s input theory (El- lis, 2003), both positive and negative input are required in learning a second language. Positive input provides grammatically correct and accept- able models of the language. Negative input is comprised of incorrect sentences that are made by non-native speakers. It teaches learners the sen- tences they should not produce. Learners’ writing skills are improved by exposure to both. A sys- tem to organize both correct sentences (for pos- itive evidence) and incorrect sentences that lan- guage learners are likely to produce (for negative evidence) would benefit language learners consid- erably. To master a foreign language, it is very effective to see where a problem lies and what caused it, rather than merely learning the …

Content truncated to 500 words