Generating Diverse Corrections with Local Beam Search for Grammatical Error Correction

Proceedings of the 28th International Conference on Computational Linguistics, pages 2132–2137 Barcelona, Spain (Online), December 8-13, 2020 2132 Generating Diverse Corrections with Local Beam Search for Grammatical Error Correction Kengo Hotate, Masahiro Kaneko, and Mamoru Komachi Tokyo Metropolitan University, Tokyo, Japan {hotate-kengo, kaneko-masahiro}@ed.tmu.ac.jp komachi@tmu.ac.jp Abstract In this study, we propose a beam search method to obtain diverse outputs in a local sequence transduction task where most of the tokens in the source and target sentences overlap, such as in grammatical error correction (GEC). In GEC, it is advisable to rewrite only the local sequences that must be rewritten while leaving the correct sequences unchanged. However, existing meth- ods of acquiring various outputs focus on revising all tokens of a sentence. Therefore, existing methods may either generate ungrammatical sentences because they force the entire sentence to be changed or produce non-diversified sentences by weakening the constraints to avoid gener- ating ungrammatical sentences. Considering these issues, we propose a method that does not rewrite all the tokens in a text, but only rewrites those parts that need to be diversely corrected. Our beam search method adjusts the search token in the beam according to the probability that the prediction is copied from the source sentence. The experimental results show that our pro- posed method generates more diverse corrections than existing methods without losing accuracy in the GEC task. 1 Introduction Grammatical error correction (GEC) is a task that corrects grammatical errors in an input text. Depending on the input, there are multiple ways to correct such text. For example, 10 annotators can produce 10 different valid correction results for the same grammatically incorrect text (Bryant and Ng, 2015). If a GEC model presents multiple candidates for correction, it helps the user decide whether to utilize the correction results such that the user can select a favorite correct expression from among the candidates. However, currently existing GEC models do not consider the generation of multiple correction can- didates. Generally, in GEC, the method for obtaining multiple corrections involves the use of a plain beam search to generate the n-best candidates (Grundkiewicz et al., 2019; Kaneko et al., 2020). How- ever, it has been shown that a plain beam search does not provide a great enough variety of candidates and produces lists of nearly identical sequences (Vijayakumar et al., 2018). Therefore, the n-best candi- dates generated by a beam search without diversity control are not expected to provide useful additional information. Considering this problem, several beam search methods have been proposed to generate diverse candidates (Li et al., 2016; Vijayakumar et al., 2018; Kulikov et al., 2019). These diverse beam search methods encourage diversity by globally rewriting all tokens in a sentence. We will refer to such methods as diverse global beam search methods. Conversely, considering a local sequence transduction task in GEC, wherein most of the tokens in the source and target sentences overlap, excessive correction of the input sentence is not preferred because unnecessary rewriting damages the grammatically correct parts of the …

Content truncated to 500 words