Grammatical Error Correction: Machine Translation and Classifiers

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2205–2215, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Grammatical Error Correction: Machine Translation and Classifiers Alla Rozovskaya Department of Computer Science Virginia Tech Blacksburg, VA 24060 alla@vt.edu Dan Roth Department of Computer Science University of Illinois Urbana, IL 61820 danr@illinois.edu Abstract We focus on two leading state-of-the-art approaches to grammatical error correc- tion – machine learning classification and machine translation. Based on the com- parative study of the two learning frame- works and through error analysis of the output of the state-of-the-art systems, we identify key strengths and weaknesses of each of these approaches and demonstrate their complementarity. In particular, the machine translation method learns from parallel data without requiring further lin- guistic input and is better at correcting complex mistakes. The classification ap- proach possesses other desirable charac- teristics, such as the ability to easily gener- alize beyond what was seen in training, the ability to train without human-annotated data, and the flexibility to adjust knowl- edge sources for individual error types. Based on this analysis, we develop an algorithmic approach that combines the strengths of both methods. We present several systems based on resources used in previous work with a relative improve- ment of over 20% (and 7.4 F score points) over the previous state-of-the-art. 1 Introduction For the majority of English speakers today, En- glish is not the first language. These writers make a variety of grammar and usage mistakes that are not addressed by standard proofing tools. Re- cently, there has been a spike in research on gram- matical error correction (GEC), correcting writing mistakes made by learners of English as a Sec- ond Language, including four shared tasks: HOO (Dale and Kilgarriff, 2011; Dale et al., 2012) and System Method Performance P R F0.5 CoNLL-2014 top 3 MT 41.62 21.40 35.01 CoNLL-2014 top 2 Classif. 41.78 24.88 36.79 CoNLL-2014 top 1 MT, rules 39.71 30.10 37.33 Susanto et al. (2014) MT, classif. 53.55 19.14 39.39 Miz. & Mats. (2016) MT 45.80 26.60 40.00 This work MT, classif. 60.17 25.64 47.40 Table 1: (Lack of) progress in GEC over the last few years. CoNLL (Ng et al., 2013; Ng et al., 2014). These shared tasks facilitated progress on the problem within the framework of two leading methods – machine learning classification and statistical ma- chine translation (MT). The top CoNLL system combined a rule-based module with MT (Felice et al., 2014). The second system that scored almost as highly used machine learning classification (Rozovskaya et al., 2014), and the third system used MT (Junczys-Dowmunt and Grundkiewicz, 2014). Furthermore, Susanto et al. (2014) showed that a combination of the two methods is beneficial, but the advantages of each method have not been fully exploited. Despite success of various methods and the growing interest in the task, the key differences be- tween the leading approaches have not been iden- tified or made explicit, which could explain the lack of progress on the …

Content truncated to 500 words