Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5441–5452 June 6–11, 2021. ©2021 Association for Computational Linguistics 5441 Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction Zhenghao Liu1,2, Xiaoyuan Yi1,2, Maosong Sun1,3∗, Liner Yang4, Tat-Seng Chua5 1Department of Computer Science and Technology, Tsinghua University, Beijing, China Institute for Artificial Intelligence, Tsinghua University, Beijing, China Beijing National Research Center for Information Science and Technology 2State Key Lab on Intelligent Technology and Systems, Tsinghua University, Beijing, China 3Beijing Academy of Artificial Intelligence 4Beijing Language and Culture University, Beijing, China 5School of Computing, National University of Singapore, Singapore Abstract Grammatical Error Correction (GEC) aims to correct writing errors and help language learn- ers improve their writing skills. However, existing GEC models tend to produce spuri- ous corrections or fail to detect lots of er- rors. The quality estimation model is neces- sary to ensure learners get accurate GEC re- sults and avoid misleading from poorly cor- rected sentences. Well-trained GEC models can generate several high-quality hypotheses through decoding, such as beam search, which provide valuable GEC evidence and can be used to evaluate GEC quality. However, ex- isting models neglect the possible GEC ev- idence from different hypotheses. This pa- per presents the Neural Verification Network (VERNet) for GEC quality estimation with multiple hypotheses. VERNet establishes in- teractions among hypotheses with a reason- ing graph and conducts two kinds of atten- tion mechanisms to propagate GEC evidence to verify the quality of generated hypotheses. Our experiments on four GEC datasets show that VERNet achieves state-of-the-art gram- matical error detection performance, achieves the best quality estimation results, and signifi- cantly improves GEC performance by rerank- ing hypotheses. All data and source codes are available at https://github.com/ thunlp/VERNet. 1 Introduction Grammatical Error Correction (GEC) systems pri- marily aim to serve second-language learners for proofreading. These systems are expected to detect grammatical errors, provide precise corrections, and guide learners to improve their language abil- ity. With the rapid increase of second-language learners, GEC has drawn growing attention from numerous researchers of the NLP community. ∗Corresponding author: M. Sun (sms@tsinghua.edu.cn) 0 0.2 0.4 0.6 0.8 1 2 3 4 5 Ratio Ranks in Beam Search Loss Win Tie (a) BEA19. 0 0.2 0.4 0.6 0.8 1 2 3 4 5 Ratio Ranks in Beam Search Loss Win Tie (b) CoNLL2014. Figure 1: The Grammaticality of Generated Hypothe- ses. The hypotheses are generated by Kiyono et al. (2019) with beam search decoding. The hypothesis is compared to the source sentence with a BERT based language model and classified into Win (the hypothe- sis is better), Tie (the hypothesis and source are same) and Loss (the source is better). The ratios of different classes are plotted with different beam search ranks. Existing GEC systems usually inherit the seq2seq architecture (Sutskever et al., 2014) to correct grammatical errors or improve sentence fluency. These systems employ beam search decod- ing to generate correction hypotheses and rerank hypotheses …

Content truncated to 500 words