Robust Systems for Preposition Error Correction Using Wikipedia Revisions

Proceedings of NAACL-HLT 2013, pages 507–517, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Robust Systems for Preposition Error Correction Using Wikipedia Revisions Aoife Cahill∗, Nitin Madnani∗, Joel Tetreault† and Diane Napolitano∗ ∗ Educational Testing Service, 660 Rosedale Road, Princeton, NJ 08541, USA {acahill, nmadnani, dnapolitano}@ets.org † Nuance Communications, Inc., 1198 E. Arques Ave, Sunnyvale, CA 94085, USA Joel.Tetreault@nuance.com Abstract We show that existing methods for training preposition error correction systems, whether using well-edited text or error-annotated cor- pora, do not generalize across very differ- ent test sets. We present a new, large error- annotated corpus and use it to train systems that generalize across three different test sets, each from a different domain and with differ- ent error characteristics. This new corpus is automatically extracted from Wikipedia revi- sions and contains over one million instances of preposition corrections. 1 Introduction One of the main themes that has defined the field of automatic grammatical error correction has been the availability of error-annotated learner data to train and test a system. Some errors, such as determiner- noun number agreement, are easily corrected us- ing rules and regular expressions (Leacock et al., 2010). On the other hand, errors involving the usage of prepositions and articles are influenced by sev- eral factors including the local context, the prior dis- course and semantics. These errors are better han- dled by statistical models which potentially require millions of training examples. Most statistical approaches to grammatical error correction have used one of the following training paradigms: 1) training solely on examples of cor- rect usage (Han et al., 2006); 2) training on exam- ples of correct usage and artificially generated er- rors (Rozovskaya and Roth, 2010); and 3) training on examples of correct usage and real learner er- rors (Dahlmeier and Ng, 2011; Dale et al., 2012). The latter two methods require annotated corpora of errors, and while they have shown great promise, manually annotating grammatical errors in a large enough corpus of learner writing is often a costly and time-consuming endeavor. In order to efficiently and automatically acquire a very large corpus of annotated learner errors, we in- vestigate the use of error corrections extracted from Wikipedia revision history. While Wikipedia re- vision history has shown promise for other NLP tasks including paraphrase generation (Max and Wisniewski, 2010; Nelken and Yamangil, 2008) and spelling correction (Zesch, 2012), this resource has not been used for the task of grammatical error cor- rection. To evaluate the usefulness of Wikipedia revision history for grammatical error correction, we address the task of correcting errors in preposition selection (i.e., where the context licenses the use of a prepo- sition, but the writer selects the wrong one). We first train a model directly on instances of correct and incorrect preposition usage extracted from the Wikipedia revision data. We also generate artificial errors using the confusion distributions derived from this data. We compare both of these approaches to models trained on well-edited text and evaluate each on three test sets …

Content truncated to 500 words