Grammatical error correction using neural machine translation

Proceedings of NAACL-HLT 2016, pages 380–386, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Grammatical error correction using neural machine translation Zheng Yuan and Ted Briscoe The ALTA Institute Computer Laboratory University of Cambridge {zy249,ejb}@cam.ac.uk Abstract This paper presents the first study using neu- ral machine translation (NMT) for grammati- cal error correction (GEC). We propose a two- step approach to handle the rare word problem in NMT, which has been proved to be useful and effective for the GEC task. Our best NMT- based system trained on the CLC outperforms our SMT-based system when testing on the publicly available FCE test set. The same sys- tem achieves an F0.5 score of 39.90% on the CoNLL-2014 shared task test set, outperform- ing the state-of-the-art and demonstrating that the NMT-based GEC system generalises ef- fectively. 1 Introduction Grammatical error correction (GEC) is the task of detecting and correcting grammatical errors in text written by non-native English writers. Un- like building machine learning classifiers for spe- cific error types (e.g. determiner or preposition er- rors) (Tetreault and Chodorow, 2008; Rozovskaya and Roth, 2011; Dahlmeier and Ng, 2011), the idea of ‘translating’ a grammatically incorrect sen- tence into a correct one has been proposed to handle all error types simultaneously (Felice et al., 2014; Junczys-Dowmunt and Grundkiewicz, 2014). Sta- tistical machine translation (SMT) has been suc- cessfully used for GEC, as demonstrated by the top-performing systems in the CoNLL-2014 shared task (Ng et al., 2014). Recently, several neural machine translation (NMT) models have been developed with promis- ing results (Kalchbrenner and Blunsom, 2013; Cho et al., 2014; Sutskever et al., 2014; Bahdanau et al., 2014). Unlike SMT, which consists of com- ponents that are trained separately and combined during decoding (i.e. the translation model and lan- guage model) (Koehn, 2010), NMT learns a single large neural network which inputs a sentence and outputs a translation. NMT is appealing for GEC as it may be possible to correct erroneous word phrases and sentences that have not been seen in the train- ing set more effectively (Luong et al., 2015). NMT- based systems thus may help ameliorate the lack of large error-annotated learner corpora for GEC. However, NMT models typically limit vocabu- lary size on both source and target sides due to the complexity of training (Sutskever et al., 2014; Bah- danau et al., 2014; Luong et al., 2015; Jean et al., 2015). Therefore, they are unable to translate rare words, and out-of-vocabulary (OOV) words are re- placed with UNK symbol. This problem is more se- rious for GEC as non-native text contains not only rare words (e.g. proper nouns), but also misspelled words (i.e. spelling errors). By replacing all the OOV words with the same UNK symbol, useful in- formation is discarded, resulting in systems that are not able to correct misspelled words or even keep some of the error-free original words, as in the fol- lowing examples (OOV words are underlined): Original sentence ... I am goign to make a …

Content truncated to 500 words