Position Offset Label Prediction for Grammatical Error Correction

Proceedings of the 29th International Conference on Computational Linguistics, pages 5409–5418 October 12–17, 2022. 5409 Position Offset Label Prediction for Grammatical Error Correction Xiuyu Wu1,2 Jingsong Yu1,2 Xu Sun1,3 Yunfang Wu1,3∗ 1MOE, Key Laboratory of Computational Linguistics, Peking University 2School of Software and Microelectronics, Peking University 3School of Computer Science, Peking University {xiuyu_wu,yujingsong,xusun,wuyf}@pku.edu.cn Abstract We introduce a novel position offset label pre- diction subtask to the encoder-decoder archi- tecture for grammatical error correction (GEC) task. To keep the meaning of the input sen- tence unchanged, only a few words should be inserted or deleted during correction, and most of tokens in the erroneous sentence appear in the paired correct sentence with limited posi- tion movement. Inspired by this observation, we design an auxiliary task to predict position offset label (POL) of tokens, which is naturally capable of integrating different correction edit- ing operations into a unified framework. Based on the predicted POL, we further propose a new copy mechanism (P-copy) to replace the vanilla copy module. Experimental results on Chinese, English and Japanese datasets demon- strate that our proposed POL-Pc framework ob- viously improves the performance of baseline models. Moreover, our model yields consistent performance gain over various data augmenta- tion methods. Especially, after incorporating synthetic data, our model achieves a 38.95 F0.5 score on Chinese GEC dataset, which outper- forms the previous state-of-the-art by a wide margin of 1.98 points. 1 Introduction Grammatical error correction (GEC) is to auto- matically correct grammatical errors in the input sequence, which is significant for both academic research and practical applications. Comparing with English grammatical error correction, Chinese grammatical error correction (CGEC) is less ad- dressed. After convolutional neural network and recurrent neural network, the Transformer based sequence- to-sequence models have achieved remarkable per- formance on GEC task (Junczys-Dowmunt et al., 2018; Kaneko et al., 2020). To alleviate the data sparsity problem, the method of generating ∗*Corresponding author. (a) Example I: deletion and substitution (b) Example II: re-ordering and insertion Figure 1: Examples of error detection and correction for CGEC task. Label @ represents a blank position. pseudo data is intensively studied (Xie et al., 2018; Lichtarge et al., 2019). Several models simplify the GEC task from sequence generation to token- level edit prediction. For example, LaserTagger (Malmi et al., 2019) predicts three edit tags Keep, Delete and Append_# for each token in the source sentence, and GECToR (Omelianchuk et al., 2020) designs ample English-specific tags to predict. To keep the meaning of the erroneous sentence unchanged, only a few words could be inserted or deleted while most words remain the same. Taking the CGEC task NLPCC data (Zhao et al., 2018) as an example: (1) about 95.6% tokens in erroneous sentences appear in their corresponding correct sen- tences (namely "cue tokens" for brevity). (2) the average lengths of erroneous sentences and the longest common sub-sequence of error-corrected sentence pairs are 29.65/27.55 tokens, indicating that cue tokens are almost in the same order. (3) the average lengths of inserted and deleted spans are …

Content truncated to 500 words