Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation

Proceedings of the 28th International Conference on Computational Linguistics, pages 2202–2212 Barcelona, Spain (Online), December 8-13, 2020 2202 Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation Zhaohong Wan1,2 and Xiaojun Wan1,2 and Wenguang Wang3 1Wangxuan Institute of Computer Technology, Peking University 2The MOE Key Laboratory of Computational Linguistics, Peking University 3DataGrand Tech Inc. {xmwzh,wanxiaojun}@pku.edu.cn Abstract The incorporation of data augmentation method in grammatical error correction task has attracted much attention. However, existing data augmentation methods mainly apply noise to tokens, which leads to the lack of diversity of generated errors. In view of this, we propose a new data augmentation method that can apply noise to the latent representation of a sentence. By editing the latent representations of grammatical sentences, we can generate synthetic samples with var- ious error types. Combining with some pre-defined rules, our method can greatly improve the performance and robustness of existing grammatical error correction models. We evaluate our method on public benchmarks of GEC task and it achieves the state-of-the-art performance on CoNLL-2014 and FCE benchmarks. 1 Introduction Grammatical Error Correction (GEC) is a task of detecting and correcting grammatical errors in a sen- tence. Due to the growing number of language learners of English, there has been increasing attention to the English GEC in the past few years. Considering the outstanding performance of neural network models in machine translation tasks, many studies have tackled GEC as a machine translation task. They regard ungrammatical sentences as the source language and grammatical sentences as the target language. This approach allows cutting-edge neural machine translation models to be applied to GEC. Many encoder-decoder models, such as recur- rent neural network (RNN)(Graves et al., 2013), convolutional neural networks (CNN)(Kim, 2014), have been widely applied to GEC. A challenge in applying neural machine translation models to GEC is the requirement of a large amount of training data, i.e., the source-target pairs. To address this problem, many data augmentation methods have been proposed. Existing methods, however, are often only able to generate sentences with limited error types, and can only improve the performance of GEC model on these few error types, while it is still hard for the model to correct the sentences with other types of errors. To address the above problem, we propose a new data augmentation method to generate synthetic samples by editing the latent representations of grammatical sentences. Given a target grammatical error type and the corresponding grammatical error type classifier, we can get a perturbation vector in latent space. Then we add the perturbation vector to the latent representation of input sentence, and use a decoder to generate a sentence with target grammatical error type. In this way, diverse errors can be generated by assigning different target error types. To further improve the performance, we adopt some rules to assist the generation of some local grammatical errors, such as spelling errors, wrong punctuation, etc. We apply this data augmentation method to the existing GEC model Copy-transformer (Zhao et al., 2019) to evaluate …

Content truncated to 500 words