System Combination for Grammatical Error Correction Based on Integer Programming

Proceedings of Recent Advances in Natural Language Processing, pages 824–829 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_094 824 System Combination for Grammatical Error Correction Based on Integer Programming Ruixi Lin Hwee Tou Ng Department of Computer Science National University of Singapore {ruixi,nght}@comp.nus.edu.sg Abstract In this paper, we propose a system combi- nation method for grammatical error correc- tion (GEC), based on nonlinear integer pro- gramming (IP). Our method optimizes a novel F score objective based on error types, and combines multiple end-to-end GEC systems. The proposed IP approach optimizes the se- lection of a single best system for each gram- matical error type present in the data. Ex- periments of the IP approach on combining state-of-the-art standalone GEC systems show that the combined system outperforms all stan- dalone systems. It improves F0.5 score by 3.61% when combining the two best participat- ing systems in the BEA 2019 shared task, and achieves F0.5 score of 73.08%. We also per- form experiments to compare our IP approach with another state-of-the-art system combina- tion method for GEC, demonstrating IP’s com- petitive combination capability. 1 Introduction Grammatical Error Correction (GEC) is the task of detecting and correcting grammatical errors of all types present in sentences in an essay, and generat- ing a corrected essay (Ng et al., 2014). Most of the latest GEC systems rely on pre- training with synthetic data and fine-tuning with task-specific data, and employ deep neural net- works with attention mechanisms (Bryant et al., 2019). Single GEC systems can be highly effec- tive in capturing a wide range of grammatical error types (Ng et al., 2014), but each individual system differs in its strengths and weaknesses in correcting certain error types, and the differences could result from the synthetic data used in pre-training a GEC system (Bryant et al., 2019). Two main categories of synthetic data generation approaches have been introduced, including directly injecting noise into grammatically correct sentences according to error distributions (Zhao et al., 2019; Choe et al., 2019), or by back-translation (Sennrich et al., 2016; Xie et al., 2018; Kiyono et al., 2019). While both cat- egories could help a system achieve a high recall across many error types, it is hard to obtain a single uniform GEC system that is good at correcting all error types. Presented with this difficulty and the strengths of individual systems, combining single GEC systems is thus a promising and efficient way to further im- prove precision and recall. Creating an ensemble of multiple systems is a common approach when it comes to combining multiple models, and the work of Junczys-Dowmunt et al. (2018) has shown its effectiveness when combining single GEC models with different random initializations and configura- tions. However, this mode of combination requires altering the component systems to achieve a tight integration. In contrast, we focus on combination methods that need to consider only the outputs of individual systems. State-of-the-art system combination ap- proaches working in this manner include Susanto et al. (2014) and Kantor et al. (2019). The ap- proach …

Content truncated to 500 words