CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction

Findings of the Association for Computational Linguistics: ACL 2022, pages 3008 - 3018 May 22-27, 2022 c⃝2022 Association for Computational Linguistics CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction Shulin Liu†, Shengkang Song†,‡, Tianchi Yue†, Tao Yang†, Huihui Cai†, Tinghao Yu†, Shengli Sun‡ †Tencent AI Platform Department, China ‡Peking University, China {forestliu,tianchiyue}@tencent.com songsk@stu.pku.edu.cn,slsun@pku.edu.cn Abstract Chinese spelling correction (CSC) models de- tect and correct a typo in texts based on the misspelled character and its context. Re- cently, Bert-based models have dominated the research of Chinese spelling correction (CSC). These methods have two limitations: (1) they have poor performance on multi-typo texts. In such texts, the context of each typo contains at least one misspelled character, which brings noise information. Such noisy context leads to the declining performance on multi-typo texts. (2) they tend to overcorrect valid expres- sions to more frequent expressions due to the masked token recovering task of Bert. We at- tempt to address these limitations in this paper. To make our model robust to contextual noise brought by typos, our approach first constructs a noisy context for each training sample. Then the correction model is forced to yield similar outputs based on the noisy and original con- texts. Moreover, to address the overcorrection problem, copy mechanism is incorporated to encourage our model to prefer to choose the input character when the miscorrected and in- put character are both valid according to the given context. Experiments are conducted on widely used benchmarks. Our model achieves superior performance against state-of-the-art methods by a remarkable gain. We release the source code and pre-trained model for further use by the community1. 1 Introduction Chinese spelling correction (CSC) is a task to de- tect and correct spelling errors in texts(Chen et al., 2013; Yu and Li, 2014). It is a challenging yet important task, which plays an important role in various NLP applications such as optical charac- ter recognition(Afli et al., 2016; Dong and Smith, 2018), automatic speech recognition(Sarma and Palmer, 2004; Errattahi et al., 2018) and search engine(Martins and Silva, 2004; Gao et al., 2010). 1https://github.com/liushulinle/CRASpell ID Text Correction E1 我是你得(de)学生。 的(de) E2 他很少座(zuo)捷运粗(cu)去玩。 坐(zuo),出(chu) Table 1: Examples of Chinese spelling errors, where error characters are marked in red and their phonics are given in brackets. Corpus Sentence-level Character-level me te ratio me te ratio S13 200 969 21% 450 1,219 37% S14 155 535 29% 406 766 53% S15 121 550 22% 284 704 40% Table 2: The multi-typo statistics of SIGHAN13, SIGHAN14 and SIGHAN15, where me denotes the number of multi-typo sentences, te denotes the num- ber of total misspelled sentences, ratio = me te . The character-level me denotes the amount of misspelled characters in multi-typo sentences. In Chinese, spelling error is mainly caused by the misuse of phonologically and visually similar char- acters(Liu et al., 2021). According to Liu et al. (2010), about 83% of errors are related to phono- logical similarity and 48% are related to visual similarity. Table1 illustrates two examples. In …

Content truncated to 500 words