Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7162–7169, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7162 Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction Mengyun Chen1∗† Tao Ge2† Xingxing Zhang2 Furu Wei2 Ming Zhou2 1 East China Normal University; 2 Microsoft Research Asia dreamcloud.chen@gmail.com; {tage,xizhang,fuwei,mingzhou}@microsoft.com Abstract We propose a novel language-independent ap- proach to improve the efficiency for Gram- matical Error Correction (GEC) by dividing the task into two subtasks: Erroneous Span Detection (ESD) and Erroneous Span Correc- tion (ESC). ESD identifies grammatically in- correct text spans with an efficient sequence tagging model. Then, ESC leverages a seq2seq model to take the sentence with annotated erro- neous spans as input and only outputs the cor- rected text for these spans. Experiments show our approach performs comparably to conven- tional seq2seq approaches in both English and Chinese GEC benchmarks with less than 50% time cost for inference. 1 Introduction Due to a growing number of error-corrected paral- lel sentences available in recent years, sequence- to-sequence (seq2seq) models with the encoder- decoder architecture (Bahdanau et al., 2014; Sutskever et al., 2014; Luong et al., 2015) have become a popular solution to GEC, which take the source (original) sentence as input and out- put the target (corrected) sentence. Although auto- regressive seq2seq models facilitate correction for various grammatical errors and perform well, they are not efficient enough for GEC. As previous work (Zhao et al., 2019) points out, seq2seq models take most decoding steps to copy grammatically correct text spans from the source to the target during in- ference, which is the main efficiency bottleneck. If the time for the copying operations can be saved, the efficiency should be much improved. With this motivation, we propose a simple yet novel language-independent approach to improve the efficiency of GEC by dividing the task into ∗ This work was done during the author’s internship at Microsoft Research Asia. †Co-first authors with equal contributions. two subtasks: Erroneous Span Detection (ESD) and Erroneous Span Correction (ESC), shown in Figure 1. In ESD, we use an efficient sequence tagging model to identify the text spans that are grammatically incorrect in the source sentence, as Figure 1(a) shows. Then, we feed the sentence with erroneous span annotations to a seq2seq model for ESC. In contrast to conventional seq2seq ap- proaches correcting the complete sentence, ESC only corrects the erroneous spans (see Figure 1(b)), which largely decreases the number of steps for decoding. Experiments in both English and Chi- nese GEC benchmarks demonstrate our approach performs comparably to the state-of-the-art trans- former based seq2seq model with less than 50% time cost for inference. Furthermore, our approach offers more flexibility to control correction, allow- ing us to adapt the precision-recall trade-off to var- ious application scenarios. 2 Related Work Recently, many approaches have been proposed to improve GEC performance. However, except those adding synthetic erroneous data (Xie et al., 2018; Ge et al., 2018a; Grundkiewicz et al., 2019; Kiyono …

Content truncated to 500 words