Data Augmentation of Incorporating Real Error Patterns and Linguistic Knowledge for Grammatical Error Correction

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 223–233 November 10–11, 2021. ©2021 Association for Computational Linguistics 223 Data Augmentation of Incorporating Real Error Patterns and Linguistic Knowledge for Grammatical Error Correction Xia Li and Junyi He Guangzhou Key Laboratory of Multilingual Intelligent Processing, School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou, China xiali@gdufs.edu.cn Abstract Data augmentation aims at expanding training data with clean text using noising schemes to improve the performance of grammatical er- ror correction (GEC). In practice, there are a great number of real error patterns in the man- ually annotated training data. We argue that these real error patterns can be introduced into clean text to effectively generate more real and high quality synthetic data, which is not fully explored by previous studies. Moreover, we also find that linguistic knowledge can be in- corporated into data augmentation for generat- ing more representative and more diverse syn- thetic data. In this paper, we propose a novel data augmentation method that fully considers the real error patterns and the linguistic knowl- edge for the GEC task. We conduct extensive experiments on public data sets and the exper- imental results show that our method outper- forms several strong baselines with far less ex- ternal unlabeled clean text data, highlighting its extraordinary effectiveness in the GEC task that lacks large-scale labeled training data. 1 Introduction Grammatical Error Correction (GEC) is the task of automatically detecting and correcting grammati- cal errors of texts into natural and correct forms, which is an important topic in the field of education, especially in language learning. In recent years, different methods have been proposed to improve the performance of the GEC models. Some early studies (Gamon et al., 2008; Tetreault et al., 2010; Dahlmeier and Ng, 2011; Berend et al., 2013; Rozovskaya and Roth, 2014) take GEC as a classification task and rely much on hand-crafted rules. More recently, the technique of neural machine translation is applied to GEC and has made remarkable performance (Zhao et al., 2019; Awasthi et al., 2019; Kiyono et al., 2019; Kaneko et al., 2020; Omelianchuk et al., 2020). Due to insufficient parallel training data publicly Figure 1: An example of real error patterns extracted from existing training data with annotations in M2 for- mat. available, the GEC task is often considered as a low-resource sequence generation task (Junczys- Dowmunt et al., 2018). In order to address this issue, many data augmentation methods have been proposed for generating synthetic pseudo training data to improve the performance of GEC. Some works focus on the edit-based approach. For exam- ple, Zhao et al. (2019) use four basic edit operations for error introduction, which are random deletion, random insertion, random substitution and random shuffling. Takahashi et al. (2020) use more fine- grained noising strategies by replacing the words in a sentence with those of the same type randomly chosen from the dictionary. Some works focus on the back-translation-based method, which attempt to train a sequence-to-sequence …

Content truncated to 500 words