Multi-Class Grammatical Error Detection for Correction: A Tale of Two Systems

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8722–8736 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8722 Multi-Class Grammatical Error Detection for Correction: A Tale of Two Systems Zheng Yuan Shiva Taslimipoor Christopher Davis Christopher Bryant ALTA Institute Department of Computer Science and Technology University of Cambridge, United Kingdom {firstname.lastname}@cl.cam.ac.uk Abstract In this paper, we show how a multi-class gram- matical error detection (GED) system can be used to improve grammatical error correction (GEC) for English. Specifically, we first de- velop a new state-of-the-art binary detection system based on pre-trained ELECTRA, and then extend it to multi-class detection using different error type tagsets derived from the ERRANT framework. Output from this detec- tion system is used as auxiliary input to fine- tune a novel encoder-decoder GEC model, and we subsequently re-rank the N-best GEC out- put to find the hypothesis that most agrees with the GED output. Results show that fine-tuning the GEC system using 4-class GED produces the best model, but re-ranking using 55-class GED leads to the best performance overall. This suggests that different multi-class GED systems benefit GEC in different ways. Ulti- mately, our system outperforms all other pre- vious work that combines GED and GEC, and achieves a new single-model NMT-based state of the art on the BEA-test benchmark. 1 Introduction Grammatical error detection (GED) is the task of automatically detecting grammatical errors in text, while grammatical error correction (GEC) is the task of also correcting these errors. Both tasks have obvious pedagogical applications that can benefit both teachers and students in online language learn- ing. GED is typically cast as a binary sequence la- belling task, where each token is classified as either correct or incorrect (Rei and Yannakoudakis, 2016; Bell et al., 2019), while GEC is often considered a sequence-to-sequence translation task, where sys- tems learn to “translate” an ungrammatical input sentence to a grammatical output sentence (Yuan and Briscoe, 2016; Junczys-Dowmunt et al., 2018; Kiyono et al., 2019; Lichtarge et al., 2020). Recent work has also begun to treat GEC as a sequence la- belling task, where tokens are classified in terms of edit operations (Awasthi et al., 2019; Omelianchuk et al., 2020). We similarly treat GED as a sequence labelling task and GEC as a sequence-to-sequence task, but additionally investigate different ways to combine and extend both approaches. In particular, we first experiment with pre- trained language models, and show that simply fine- tuning ELECTRA (Clark et al., 2020) leads to sig- nificant improvements in binary GED and achieves a new state of the art. Given that binary detection is limited in terms of the specific error type informa- tion it can provide to downstream tasks however, we also extend our GED system to 4-class, 25-class and 55-class error detection using different error type tagsets derived from the ERRANT framework (Bryant et al., 2017). To illustrate the value of multi-class GED for downstream GEC, we extend the Transformer encoder-decoder model (Vaswani et al., 2017) and employ …

Content truncated to 500 words