Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6713–6742 July 9-14, 2023 ©2023 Association for Computational Linguistics Towards standardizing Korean Grammatical Error Correction: Datasets and Annotation Soyoung Yoon KAIST AI soyoungyoon@kaist.ac.kr Sungjoon Park SoftlyAI Research, SoftlyAI sungjoon.park@softly.ai Gyuwan Kim UCSB∗ gyuwankim@ucsb.edu Junhee Cho Google junheecho@google.com Kihyo Park Cornell University kp467@cornell.edu Gyutae Kim SoftlyAI gt.kim@softly.ai Minjoon Seo KAIST AI minjoon@kaist.ac.kr Alice Oh KAIST SC† alice.oh@kaist.edu Abstract Research on Korean grammatical error correc- tion (GEC) is limited, compared to other ma- jor languages such as English. We attribute this problematic circumstance to the lack of a carefully designed evaluation benchmark for Korean GEC. In this work, we collect three datasets from different sources (Kor-Lang8, Kor-Native, and Kor-Learner) that covers a wide range of Korean grammatical errors. Con- sidering the nature of Korean grammar, We then define 14 error types for Korean and pro- vide KAGAS (Korean Automatic Grammati- cal error Annotation System), which can au- tomatically annotate error types from paral- lel corpora. We use KAGAS on our datasets to make an evaluation benchmark for Korean, and present baseline models trained from our datasets. We show that the model trained with our datasets significantly outperforms the cur- rently used statistical Korean GEC system (Hanspell) on a wider range of error types, demonstrating the diversity and usefulness of the datasets. The implementations and datasets are open-sourced.1 1 Introduction Writing grammatically correct Korean sentences is difficult for learners studying Korean as a For- eign Language (KFL) and even for native Korean speakers due to its morphological and orthographi- cal complexity such as particles, spelling, and col- location. Its word spacing rule is complex since there are many domain-dependent exceptions, of which only around 20% of native speakers under- stand thoroughly (Lee, 2014). Since Korean is an agglutinative language (Sohn, 2001; Song, 2006), getting used to Korean grammar is time-consuming ∗ University of California, Santa Barbara † School of Computing, KAIST 1Code for model checkpoints and KAGAS: https: //github.com/soyoung97/Standard_Korean_GEC. Dataset request form: https://forms.gle/kF9pvJbLGvnh8ZnQ6 for KFL learners whose mother tongue is non- agglutinative (Haupt et al., 2017; Kim, 2020). How- ever, despite the growing number of KFL learners (Lee, 2018), little research has been conducted on Korean Grammatical Error Correction (GEC) be- cause of the previously described difficulties of the Korean language. Another major obstacle to devel- oping a Korean GEC system is the lack of resources to train a machine learning model. In this paper, we propose three datasets that cover various grammatical errors from different types of annotators and learners. The first dataset named Kor-Native is crowd-sourced from native Korean speakers. Second, Kor-Learner are from KFL learners that consists of essays with de- tailed corrections and annotations by Korean tutors. Third, Kor-Lang8 are similar with Kor-Learner ex- cept that they consist of sentences made by KFL learners but corrected by native Koreans on social platforms who are not necessarily linguistic experts. We also analyze our datasets in terms of error type distributions. While our proposed …

Content truncated to 500 words