Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5937–5947 August 1–6, 2021. ©2021 Association for Computational Linguistics 5937 Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding Xin Sun1∗† Tao Ge2† Furu Wei2 Houfeng Wang1 1 MOE Key Lab of Computational Linguistics, School of EECS, Peking University; 2 Microsoft Research Asia {sunx5,wanghf}@pku.edu.cn; {tage,fuwei}@microsoft.com Abstract In this paper, we propose Shallow Aggressive Decoding (SAD) to improve the online infer- ence efficiency of the Transformer for instan- taneous Grammatical Error Correction (GEC). SAD optimizes the online inference efficiency for GEC by two innovations: 1) it aggres- sively decodes as many tokens as possible in parallel instead of always decoding only one token in each step to improve computational parallelism; 2) it uses a shallow decoder in- stead of the conventional Transformer archi- tecture with balanced encoder-decoder depth to reduce the computational cost during infer- ence. Experiments in both English and Chi- nese GEC benchmarks show that aggressive decoding could yield the same predictions as greedy decoding but with a significant speedup for online inference. Its combination with the shallow decoder could offer an even higher online inference speedup over the powerful Transformer baseline without quality loss. Not only does our approach allow a single model to achieve the state-of-the-art results in English GEC benchmarks: 66.4 F0.5 in the CoNLL- 14 and 72.9 F0.5 in the BEA-19 test set with an almost 10× online inference speedup over the Transformer-big model, but also it is easily adapted to other languages. Our code is avail- able at https://github.com/AutoTemp/ Shallow-Aggressive-Decoding. 1 Introduction The Transformer (Vaswani et al., 2017) has become the most popular model for Grammatical Error Cor- rection (GEC). In practice, however, the sequence- to-sequence (seq2seq) approach has been blamed recently (Chen et al., 2020; Stahlberg and Kumar, ∗ This work was done during the author’s internship at MSR Asia. Contact person: Tao Ge (tage@microsoft.com) †Co-first authors with equal contributions 2020; Omelianchuk et al., 2020) for its poor infer- ence efficiency in modern writing assistance ap- plications (e.g., Microsoft Office Word1, Google Docs2 and Grammarly3) where a GEC model usu- ally performs online inference, instead of batch in- ference, for proactively and incrementally checking a user’s latest completed sentence to offer instanta- neous feedback. To better exploit the Transformer for instanta- neous GEC in practice, we propose a novel ap- proach – Shallow Aggressive Decoding (SAD) to improve the model’s online inference efficiency. The core innovation of SAD is aggressive decoding: instead of sequentially decoding only one token at each step, aggressive decoding tries to decode as many tokens as possible in parallel with the assump- tion that the output sequence should be almost the same with the input. As shown in Figure 1, if the output prediction at each step perfectly matches its counterpart in the input sentence, the inference will finish, meaning that the model will keep the input untouched without editing; if the output token at a step …

Content truncated to 500 words