Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2858–2869, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2858 Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples Lihao Wang, Xiaoqing Zheng School of Computer Science, Fudan University, Shanghai, China Shanghai Key Laboratory of Intelligent Information Processing {wanglh19, zhengxq}@fudan.edu.cn Abstract A sequence-to-sequence (seq2seq) learning with neural networks empirically shows to be an effective framework for grammatical error correction (GEC), which takes a sentence with errors as input and outputs the corrected one. However, the performance of GEC models with the seq2seq framework heavily relies on the size and quality of the corpus on hand. We propose a method inspired by adversarial train- ing to generate more meaningful and valuable training examples by continually identifying the weak spots of a model, and to enhance the model by gradually adding the generated ad- versarial examples to the training set. Exten- sive experimental results show that such adver- sarial training can improve both the generaliza- tion and robustness of GEC models. 1 Introduction The goal of Grammatical Error Correction (GEC) is to identify and correct different kinds of errors in the text, such as spelling, punctuation, grammatical, and word choice errors, which has been widely used in speech-based dialogue, web information extraction, and text editing software. A popular solution tackles the grammatical error correction as a monolingual machine translation task where ungrammatical sentences are regarded as the source language and corrected sentences as the target language (Ji et al., 2017; Chollampatt and Ng, 2018a). Therefore, the GEC can be mod- eled using some relatively mature machine trans- lation models, such as the sequence-to-sequence (seq2seq) paradigm (Sutskever et al., 2014). They are many types of grammatical errors and all of them can occur in a sentence, which makes it impossible to construct a corpus that covers all kinds of errors and their combinations. Deep learn- ing so far is data-hungry and it is hard to train a seq2seq model with good performance without suf- Clean His reaction should give you an idea as to whether this matter is or is not your business . Direct Noise His reaction should give you an as idea to whether this matter is or so is not your business . Back Translation His reaction should give you the idea as to whether this matter is or is not your business . Adversarial Example His reaction should gave you an idea as to whether this matter is or is n’t their business . Table 1: Example ungrammatical sentences generated by different methods. The direct noise method seems to add some meaningless noises to the original sen- tence. Most of the grammatical errors generated by the back translation are those produced by replacing prepo- sitions, and inserting or deleting articles. The exam- ple generated by our adversarial attack algorithm looks more meaningful and valuable. ficient data. Therefore, recent studies have turned the focus to the methods of generating high-quality training samples (Xie et al., 2018; Lichtarge …

Content truncated to 500 words