An Extended Sequence Tagging Vocabulary for Grammatical Error Correction

Findings of the Association for Computational Linguistics: EACL 2023, pages 1608–1619 May 2-6, 2023 ©2023 Association for Computational Linguistics An Extended Sequence Tagging Vocabulary for Grammatical Error Correction Stuart Mesham♠ Christopher Bryant♢ Marek Rei♡♢ Zheng Yuan♣♢ ♠Department of Computer Science, University of Cambridge ♢The ALTA Institute, Department of Computer Science, University of Cambridge ♡Department of Computing, Imperial College London ♣Department of Informatics, King’s College London sm2613@cantab.ac.uk, christopher.bryant@cl.cam.ac.uk marek.rei@imperial.ac.uk, zheng.yuan@kcl.ac.uk Abstract We extend a current sequence-tagging approach to Grammatical Error Correction (GEC) by introducing specialised tags for spelling cor- rection and morphological inflection using the SymSpell and LemmInflect algorithms. Our ap- proach improves generalisation: the proposed new tagset allows a smaller number of tags to correct a larger range of errors. Our results show a performance improvement both over- all and in the targeted error categories. We further show that ensembles trained with our new tagset outperform those trained with the baseline tagset on the public BEA benchmark. 1 Introduction Current approaches to Grammatical Error Cor- rection (GEC) fall under two broad categories: sequence-to-sequence and sequence-tagging. The former treats GEC as a machine-translation prob- lem, "translating" from error-containing to error- free language (Yuan and Briscoe, 2016; Schmaltz et al., 2017; Junczys-Dowmunt et al., 2018; Grund- kiewicz et al., 2019; Yuan et al., 2019; Rothe et al., 2021). By contrast, sequence-tagging approaches tag each input word with an edit operation such that applying the operations produces the corrected output (Yannakoudakis et al., 2017; Awasthi et al., 2019; Omelianchuk et al., 2020; Tarnavskyi et al., 2022). The basic operations include keeping a word unchanged, deleting a word, and inserting new words (Awasthi et al., 2019; Malmi et al., 2019). One advantage of sequence-tagging over sequence-to-sequence approaches is computational efficiency: the former do not require expensive auto-regressive decoding,1 and currently achieve competitive performance using smaller models (Tarnavskyi et al., 2022; Rothe et al., 2021). 1Malmi et al. (2019) show that sequence-taggers can be orders of magnitude faster than comparable seq-to-seq models at inference time. the serendipitis discovery of penicillin $KEEP $SPELL $KEEP $KEEP $KEEP SymSpell serendipitous It was easy than taming a dragon $KEEP $KEEP $INFLECT_JJR $KEEP $KEEP $KEEP $KEEP LemmInflect easier Figure 1: Our model applied to two inputs. Beneath each word is the tagger’s output. Arrows denote trans- formations by SymSpell and LemmInflect respectively. However, current sequence-tagging approaches re- quire manual linguistic efforts to curate language- specific edit tags (Yuan et al., 2021). For example, Awasthi et al. (2019) introduce rule-based morpho- logical inflection tags, like replacing the "-ing" suffix with "-ion" (e.g. completing → comple- tion). Omelianchuk et al. (2020) introduce a wider range of operations including verb-form and noun- number changes. For verb-form inflections, they use a dictionary to map between verb forms.2 In this paper, we focus on a sequence-tagging approach. We extend the approach of Omelianchuk et al. (2020) by introducing more general transfor- mation tags (Figure 1). Specifically, we introduce: • A tag for correcting spelling errors. • Inflection tags capable of a broader range of inflections …

Content truncated to 500 words