Cross-Sentence Grammatical Error Correction

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 435–445 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 435 Cross-Sentence Grammatical Error Correction Shamil Chollampatt, Weiqi Wang, and Hwee Tou Ng Department of Computer Science, National University of Singapore {shamil,weiqi}@u.nus.edu, nght@comp.nus.edu.sg Abstract Automatic grammatical error correction (GEC) research has made remarkable progress in the past decade. However, all existing ap- proaches to GEC correct errors by considering a single sentence alone and ignoring crucial cross-sentence context. Some errors can only be corrected reliably using cross-sentence context and models can also benefit from the additional contextual information in correct- ing other errors. In this paper, we address this serious limitation of existing approaches and improve strong neural encoder-decoder models by appropriately modeling wider contexts. We employ an auxiliary encoder that encodes previous sentences and incorporate the encoding in the decoder via attention and gating mechanisms. Our approach results in statistically significant improvements in overall GEC performance over strong baselines across multiple test sets. Analysis of our cross-sentence GEC model on a synthetic dataset shows high performance in verb tense corrections that require cross-sentence context. 1 Introduction Grammatical error correction (GEC) is the task of correcting errors in input text and produc- ing well-formed output text. GEC models are essential components of writing assistance and proof-reading tools that help both native and non-native speakers. Several adaptations of so- phisticated sequence-to-sequence learning models with specialized techniques have been shown to achieve impressive performance (Ge et al., 2018; Lichtarge et al., 2018; Chollampatt and Ng, 2018b; Junczys-Dowmunt et al., 2018). All prior approaches to GEC consider one sen- tence at a time and ignore useful contextual infor- mation from the document in which it appears, un- like a human proofreader. Cross-sentence context OUT OF CONTEXT: As a result, they are not convenient enough. IN CONTEXT: Electric cars have a very obvious shortage in their technique design. The electric cars in- vented in 1990 did not have a powerful bat- tery. Due to the limitation of its weigh, size and the battery technology, the battery used in the electric cars at that time was limited to a range of 100 miles (Rogers,2003). As a re- sult, they are were not convenient enough. Instead, Hydrogen fuel cell was brought up to substitute the electric battery. Figure 1: A sentence from NUCLE appears correct out of context, but erroneous in context. is essential to identify and correct certain errors, mostly involving tense choice, use of the definite article ‘the’, and use of connectives. The example in Figure 1, from the NUS Corpus of Learner En- glish or NUCLE (Dahlmeier et al., 2013), shows a learner-written sentence that seems correct in iso- lation, but actually involves a verb tense error in context. Sentence-level GEC systems fail to reli- ably correct such errors. Moreover, models may also benefit from the additional context by being able to disambiguate error corrections better. In this paper, we present the first approach to cross-sentence …

Content truncated to 500 words