UMRSpell: Unifying the Detection and Correction Parts of Pre-trained Models towards Chinese Missing, Redundant, and Spelling Correction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 10238–10250 July 9-14, 2023 ©2023 Association for Computational Linguistics UMRSpell: Unifying the Detection and Correction Parts of Pre-trained Models towards Chinese Missing, Redundant, and Spelling Correction Zheyu He1,†, Yujin Zhu1,†, Linlin Wang2,3, Liang Xu1,∗ 1GammaLab, PingAn OneConnect, Shanghai, China 2East China Normal University 3Shanghai Artificial Intelligence Laboratory hezheyu874@pingan.com.cn, zhuyujin204@ocft.com, llwang@cs.ecnu.edu.cn., xlpaul@126.com Abstract Chinese Spelling Correction (CSC) is the task of detecting and correcting misspelled charac- ters in Chinese texts. As an important step for various downstream tasks, CSC confronts two challenges: 1) Character-level errors consist not only of spelling errors but also of missing and redundant ones that cause variable length between input and output texts, for which most CSC methods could not handle well because of the consistence length of texts required by their inherent detection-correction framework. Con- sequently, the two errors are considered out- side the scope and left to future work, despite the fact that they are widely found and bound to CSC task in Chinese industrial scenario, such as Automatic Speech Recognition (ASR) and Optical Character Recognition (OCR). 2) Most existing CSC methods focus on either detector or corrector and train different mod- els for each one, respectively, leading to in- sufficiency of parameters sharing. To address these issues, we propose a novel model UMR- Spell to learn detection and correction parts together at the same time from a multi-task learning perspective by using a detection trans- mission self-attention matrix, and flexibly deal with both missing, redundant, and spelling er- rors through re-tagging rules. Furthermore, we build a new dataset ECMR-2023 containing five kinds of character-level errors to enrich the CSC task closer to real-world applications. Ex- periments on both SIGHAN benchmarks and ECMR-2023 demonstrate the significant effec- tiveness of UMRSpell over previous represen- tative baselines. 1 Introduction Recent decades have witnessed the comprehensive development of one important Neural Language Processing (NLP) task: Chinese Spelling Correc- tion (CSC), which focus on detecting and correct- ing spelling errors in texts (Yu and Li, 2014). The †Equal contributions. *Corresponding author. task shares a long research line originated from early 1990s (Shih et al., 1992; Chang, 1995), but remains challenging in Chinese context because that different from English, many Chinese charac- ters are phonologically and visually similar while semantically diverse (Liu et al., 2010). Moreover, CSC not only plays an essential role for various downstream tasks such as search engine (Martins and Silva, 2004; Gao et al., 2010) or automatic es- say scoring (Burstein and Chodorow, 1999), but also becomes a necessary part in some widely used industrial applications such as Automatic Speech Recognition (ASR) (Sarma and Palmer, 2004; Er- rattahi et al., 2018) and Optical Character Recogni- tion (OCR) (Afli et al., 2016; Hládek et al., 2020) systems. Early work on CSC includes but not limited to pipeline strategy (Chen et al., 2013), traditional language model (Yu and Li, 2014), and sequence- to-sequence learning (Wang et al., 2019). With the growth …

Content truncated to 500 words