Neural Sequence-Labelling Models for Grammatical Error Correction

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2795–2806 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Neural Sequence-Labelling Models for Grammatical Error Correction Helen Yannakoudakis, Marek Rei, Øistein E. Andersen and Zheng Yuan The ALTA Institute Computer Laboratory University of Cambridge {hy260,mr472,oa223,zy249}@cl.cam.ac.uk Abstract We propose an approach to N-best list re- ranking using neural sequence-labelling models. We train a compositional model for error detection that calculates the prob- ability of each token in a sentence being correct or incorrect, utilising the full sen- tence as context. Using the error detec- tion model, we then re-rank the N best hypotheses generated by statistical ma- chine translation systems. Our approach achieves state-of-the-art results on error correction for three different datasets, and it has the additional advantage of only us- ing a small set of easily computed features that require no linguistic input. 1 Introduction Grammatical Error Correction (GEC) in non- native text attempts to automatically detect and correct errors that are typical of those found in learner writing. High precision and good coverage of learner errors is important in the development of GEC systems. Phrase-based Statistical Machine Translation (SMT) approaches to GEC have at- tracted considerable attention in recent years as they have been shown to achieve state-of-the-art results (Felice et al., 2014; Junczys-Dowmunt and Grundkiewicz, 2016). Given an ungrammatical in- put sentence, the task is formulated as “translat- ing” it to its grammatical counterpart. Using a par- allel dataset of input sentences and their corrected counterparts, SMT systems are typically trained to correct all error types in text without requir- ing any further linguistic input. To further adapt SMT approaches to the task of GEC and tackle the paucity of error-annotated learner data, previ- ous work has investigated a number of extensions, ranging from the addition of further features into the decoding process (Felice et al., 2014) via re- ranking the SMT decoder’s output (Yuan et al., 2016) to neural-network adaptation components to SMT (Chollampatt et al., 2016a). In this paper, we propose an approach to N-best list re-ranking using neural sequence-labelling models. N-best list re-ranking allows for fast ex- perimentation since the decoding process remains unchanged and only needs to be performed once. Crucially, it can be applied to any GEC system that can produce multiple alternative hypotheses. More specifically, we train a neural compositional model for error detection that calculates the prob- ability of each token in a sentence being correct or incorrect, utilising the full sentence as context. Using the error detection model, we then re-rank the N best hypotheses generated by the SMT sys- tem. Detection models can be more fine-tuned to finer nuances of grammaticality and acceptability, and therefore better able to distinguish between correct and incorrect versions of a sentence. Our approach achieves state-of-the-art results on GEC for three different datasets, and it has the additional advantage of using only a small set of easily computed features that require no linguis- tic information, in contrast to previous …

Content truncated to 500 words