Joint Learning and Inference for Grammatical Error Correction

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 791–802, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Joint Learning and Inference for Grammatical Error Correction Alla Rozovskaya and Dan Roth Cognitive Computation Group University of Illinois at Urbana-Champaign 201 N. Goodwin Avenue Urbana, IL 61801 {rozovska,danr}@illinois.edu Abstract State-of-the-art systems for grammatical er- ror correction are based on a collection of independently-trained models for specific er- rors. Such models ignore linguistic interac- tions at the sentence level and thus do poorly on mistakes that involve grammatical depen- dencies among several words. In this paper, we identify linguistic structures with interact- ing grammatical properties and propose to ad- dress such dependencies via joint inference and joint learning. We show that it is possible to identify interac- tions well enough to facilitate a joint approach and, consequently, that joint methods correct incoherent predictions that independently- trained classifiers tend to produce. Further- more, because the joint learning model con- siders interacting phenomena during training, it is able to identify mistakes that require mak- ing multiple changes simultaneously and that standard approaches miss. Overall, our model significantly outperforms the Illinois system that placed first in the CoNLL-2013 shared task on grammatical error correction. 1 Introduction There has recently been a lot of work addressing er- rors made by English as a Second Language (ESL) learners. In the past two years, three competitions devoted to grammatical error correction for non- native writers took place: HOO-2011 (Dale and Kil- garriff, 2011), HOO-2012 (Dale et al., 2012), and the CoNLL-2013 shared task (Ng et al., 2013). Nowadays *phone/phones *has/have many functionalities, *included/including *∅/a camera and *∅/a Wi-Fi receiver. Figure 1: Examples of representative ESL errors. Most of the work in the area of ESL error cor- rection has addressed the task by building statistical models that specialize in correcting a specific type of a mistake. Figure 1 illustrates several types of errors common among non-native speakers of En- glish: article, subject-verb agreement, noun num- ber, and verb form. A significant proportion of re- search has focused on correcting mistakes in article and preposition usage (Izumi et al., 2003; Han et al., 2006; Felice and Pulman, 2008; Gamon et al., 2008; Tetreault and Chodorow, 2008; Gamon, 2010; Rozovskaya and Roth, 2010b). Several studies also consider verb-related and noun-related errors (Lee and Seneff, 2008; Gamon et al., 2008; Dahlmeier and Ng, 2012). The predictions made by individual models are then applied independently (Rozovskaya et al., 2011) or pipelined (Dahlmeier and Ng, 2012). The standard approach of training individual clas- sifiers considers each word independently and thus assumes that there are no interactions between er- rors and between grammatical phenomena. But an ESL writer may make multiple mistakes in a single sentence and these result in misleading local cues given to individual classifiers. In the example shown in Figure 1, the agreement error on the verb “have” interacts with the noun number error: a correction system that takes into account the context may …

Content truncated to 500 words