Adversarial Grammatical Error Correction

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3075–3087 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3075 Adversarial Grammatical Error Correction Vipul Raheja Dimitris Alikaniotis Grammarly firstname.lastname@grammarly.com Abstract Recent works in Grammatical Error Cor- rection (GEC) have leveraged the progress in Neural Machine Translation (NMT), to learn rewrites from parallel corpora of gram- matically incorrect and corrected sentences, achieving state-of-the-art results. At the same time, Generative Adversarial Networks (GANs) have been successful in generating realistic texts across many different tasks by learning to directly minimize the difference between human-generated and synthetic text. In this work, we present an adversarial learn- ing approach to GEC, using the generator- discriminator framework. The generator is a Transformer model, trained to produce gram- matically correct sentences given grammati- cally incorrect ones. The discriminator is a sentence-pair classification model, trained to judge a given pair of grammatically incorrect- correct sentences on the quality of grammat- ical correction. We pre-train both the dis- criminator and the generator on parallel texts and then fine-tune them further using a pol- icy gradient method that assigns high rewards to sentences which could be true corrections of the grammatically incorrect text. Experi- mental results on FCE, CoNLL-14, and BEA- 19 datasets show that Adversarial-GEC can achieve competitive GEC quality compared to NMT-based baselines. 1 Introduction Grammatical Error Correction (GEC) has grown into a popular NLP task that deals with building systems for automatically correcting errors in writ- ten text (Ng et al., 2013, 2014). Evolving from the approaches of building error-specific machine learning classifiers (Tetreault and Chodorow, 2008; De Felice and Pulman, 2008; Tetreault et al., 2010; Dahlmeier and Ng, 2011; Rozovskaya and Roth, 2014), it has gained popularity as a monolingual Machine Translation (MT) problem, where the sys- tem learns to “translate” a given erroneous text to its corrected form (Brockett et al., 2006; Felice et al., 2014; Susanto et al., 2014). Initially, Sta- tistical phrase-based Machine Translation (SMT) techniques were successfully applied to the task (Yuan and Felice, 2013; Junczys-Dowmunt and Grundkiewicz, 2016; Yuan et al., 2016) as as a way to handle all error types concurrently. More re- cently, several Neural Machine Translation (NMT) systems have been developed with promising re- sults (Sutskever et al., 2014; Bahdanau et al., 2015; Cho et al., 2014), and their successful application to GEC, either in combination with SMT mod- els (Chollampatt et al., 2016; Yuan and Briscoe, 2016; Yannakoudakis et al., 2017; Grundkiewicz and Junczys-Dowmunt, 2018), or strictly as neu- ral models, has emerged as the new state-of-the- art (Xie et al., 2016; Schmaltz et al., 2017; Sak- aguchi et al., 2017; Ji et al., 2017; Ge et al., 2018; Junczys-Dowmunt et al., 2018; Chollampatt and Ng, 2018a,b; Zhao et al., 2019). Despite the successes of NMT-based models for GEC, a major challenge still lies in the definition of the evaluation metrics. Ideally, the metric should be able to quantify the (a) lexical overlap, (b) semantic similarity, and (c) grammaticality of a generated sentence, given …

Content truncated to 500 words