Type-Driven Multi-Turn Corrections for Grammatical Error Correction

Findings of the Association for Computational Linguistics: ACL 2022, pages 3225 - 3236 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Type-Driven Multi-Turn Corrections for Grammatical Error Correction Shaopeng Lai1∗, Qingyu Zhou2, Jiali Zeng2, Zhongli Li2, Chao Li2, Yunbo Cao2, Jinsong Su1,3† 1School of Informatics, Xiamen University, China 2Tencent Cloud Xiaowei, China 3Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism, China splai@stu.xmu.edu.cn, {qingyuzhou, lemonzeng, neutrali, diegoli, yunbocao}@tencent.com, jssu@xmu.edu.cn Abstract Grammatical Error Correction (GEC) aims to automatically detect and correct grammatical errors. In this aspect, dominant models are trained by one-iteration learning while perform- ing multiple iterations of corrections during inference. Previous studies mainly focus on the data augmentation approach to combat the exposure bias, which suffers from two draw- backs. First, they simply mix additionally- constructed training instances and original ones to train models, which fails to help models be explicitly aware of the procedure of grad- ual corrections. Second, they ignore the in- terdependence between different types of cor- rections. In this paper, we propose a Type- Driven Multi-Turn Corrections approach for GEC. Using this approach, from each train- ing instance, we additionally construct multiple training instances, each of which involves the correction of a specific type of errors. Then, we use these additionally-constructed training in- stances and the original one to train the model in turn. By doing so, our model is trained to not only correct errors progressively, but also exploit the interdependence between different types of errors for better performance. Experi- mental results and in-depth analysis show that our approach significantly benefits the model training. Particularly, our enhanced model achieves state-of-the-art single-model perfor- mance on English GEC benchmarks. We re- lease our code at https://github.com/ DeepLearnXMU/TMTC. 1 Introduction Grammatical Error Correction (GEC) aims at au- tomatically detecting and correcting grammatical (and other related) errors in a text. It attracts much attention due to its practical applications in writ- ing assistant (Napoles et al., 2017b; Ghufron and aowei∗ Work is done during internship at Tencent Cloud Xi- † Corresponding author Rosyida, 2018), speech recognition systems (Karat et al., 1999; Wang et al., 2020; Kubis et al., 2020) etc. Inspired by the success of neural machine translation (NMT), some models adopt the same paradigm, namely NMT-based models. They have been quite successful, especially with data augmen- tation approach (Boyd, 2018; Ge et al., 2018; Xu et al., 2019; Grundkiewicz et al., 2019; Wang and Zheng, 2020; Takahashi et al., 2020). However, these models have been blamed for their ineffi- ciency during inference (Chen et al., 2020; Sun et al., 2021). To tackle this issue, many researchers resort to the sequence-to-label (Seq2Label) formu- lation, achieving comparable or better performance with efficiency (Malmi et al., 2019; Awasthi et al., 2019; Stahlberg and Kumar, 2020; Omelianchuk et al., 2020). Despite their success, both NMT-based and Seq2Label models are trained by one-iteration learning, while correcting errors for multiple it- erations during inference. As a consequence, they suffer from exposure …

Content truncated to 500 words