Global Attention Decoder for Chinese Spelling Error Correction

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1419–1428 August 1–6, 2021. ©2021 Association for Computational Linguistics 1419 Global Attention Decoder for Chinese Spelling Error Correction Zhao Guo1, Yuan Ni1, Keqiang Wang1, Wei Zhu1, Guotong Xie1,2,3 1Ping An Healthcare Technology 2Ping An Healthcare and Technology Company Limited 3Ping An International Smart City Technology Co.,Ltd {GUOZHAO385,NIYUAN442,WANGKEQIANG265,ZHUWEI972, XIEGUOTONG}@pingan.com Abstract Recent progress has been made in using BERT framework for Chinese spelling error correc- tion (CSC). However, most existing methods correct words based on local contextual in- formation, without considering the influence of error words in sentences. Imposing at- tention on error contextual information could mislead and decrease the overall performance of CSC. To address this issue, we propose a Global Attention Decoder (GAD) approach for CSC. Specifically, the proposed method learns the global relationship of the poten- tial correct input characters and the candi- dates of potential error characters. Rich global contextual information is obtained to alleviate the impact of the local error contex- tual information. In addition, a BERT with Confusion set guided Replacement Strategy (BERT CRS) is designed to narrow the gap between BERT and CSC. The candidates gen- erated by BERT CRS covering the correct character with more than 99.9% probability. To demonstrate the effectiveness of our pro- posed framework, we test our method on three human-annotated datasets. The experimental results show that our approach outperforms all competitor models by a large margin of up to 6.2%, achieving state-of-the-art methods on all datasets. 1 Introduction Spelling error correction plays an important role in NLP domain. A good spelling error sys- tem is the key to improve the performance of upper-layer applications. Spelling error correc- tion aims to detect and correct erroneous charac- ters/words. These spelling errors are mainly from human writing, speech recognition and optical char- acter recognition (OCR) (Afli et al., 2016) systems. In Chinese, erroneous type is usually from charac- ter/word’s phonological, visual and semantic sim- ilarity. According to (Cheng et al., 2020), about Input 餐厅的换经费产适合约会 The restaurant’s swap property is suitable for dates BERT CRS 餐厅的环经非常适合约会 The restaurant’s ring is perfect for dates +GAD 餐厅的环境非常适合约会 The restaurant environment is perfect for dates Table 1: A sample data from SIGHAN 2014 (Yu et al., 2014), the incorrect and correct characters marked in red and green color respectively. Since ”经” is highly related to ”费” in its context, BERT CRS is difficult to correct. GAD method learns the global relationship be- tween ”环” and ”境” in candidates of input error char- acters ”换” and ”经” respectively (see Fig.1). Rich global contextual information is learned to alleviate the impact of the local noisy contextual information. 83% and 48% of errors are related to phonolog- ical and visual similarity respectively. Although lots of researches have made great progress, Chi- nese spelling error correction (CSC) still remains a challenging task. Moreover, because the Chinese is composed of pictographic characters without word delimiters, methods from the languages like English can hardly be used for the Chinese. In ad- dition, the …

Content truncated to 500 words