Dynamic Connected Networks for Chinese Spelling Check

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2437–2446 August 1–6, 2021. ©2021 Association for Computational Linguistics 2437 Dynamic Connected Networks for Chinese Spelling Check Baoxin Wang1,2, Wanxiang Che1, Dayong Wu2, Shijin Wang2,3, Guoping Hu2, Ting Liu1 1Research Center for SCIR, Harbin Institute of Technology, Harbin, China 2State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, China 3iFLYTEK AI Research (Hebei), Langfang, China {bxwang2,dywu2,sjwang3,gphu}@iflytek.com {car,tliu}@ir.hit.edu.cn Abstract Chinese spelling check (CSC) is a task to detect and correct spelling errors in Chinese text. Most state-of-the-art works on the CSC task adopt a BERT-based non-autoregressive language model, which relies on the output independence assumption. The inappropri- ate independence assumption prevents BERT- based models from learning the dependencies among target tokens, resulting in an incoher- ent problem. To address the above issue, we propose a novel architecture named Dynamic Connected Networks (DCN), which generates the candidate Chinese characters via a Pinyin Enhanced Candidate Generator and then uti- lizes an attention-based network to model the dependencies between two adjacent Chinese characters. The experimental results show that our proposed method achieves a new state-of- the-art performance on three human-annotated datasets. 1 Introduction Chinese spelling check (CSC) is an important task which can be utilized in many natural language applications such as optical character recognition (OCR) (Wang et al., 2018; Hong et al., 2019) and essay scoring. Meanwhile, CSC is a chal- lenging task which requires human-level natural language understanding ability (Liu et al., 2010, 2013; Xin et al., 2014). Recently, BERT-based non-autoregressive language models have achieved state-of-the-art performance in the CSC task (Hong et al., 2019; Zhang et al., 2020; Cheng et al., 2020). These works fine-tune BERT-based models us- ing CSC training data. During the training phase, all the target Chinese characters will be involved as labels. In the inference stage, the models predict the most likely Chinese character from a candidate set at each position. When the most likely charac- ter is different from the input character, the input Wrong: 我忘记告诉你了，我真户秃。 Correct: 我忘记告诉你了，我真糊涂。 Translation: I forgot to tell you. I’m so confused. Table 1: An example of Chinese spelling errors. Here, “户秃” should be corrected to “糊涂” (confused). character will be considered as a spelling error and corrected to the most likely character. Based on the powerful generalization ability of BERT (De- vlin et al., 2019), these works have achieved better performance than other models. However, these works on the CSC task rely on the incorrect independence assumption, which may lead to an incoherent problem. Concretely, they as- sume that the predicted tokens are independent of each other, which generally does not hold in natural language (Yang et al., 2019; Gu and Kong, 2020). For the CSC task, one spelling error may have mul- tiple corrections. Ignoring the corrected context may result in a correction conflict. As shown in Table 1, “户秃” may be corrected as “糊涂” (con- fused) or “尴尬” (embarrassed). Because of the independence of each token, the non-autoregressive language model may correct it as “尴涂” (emba- fused). This incoherent …

Content truncated to 500 words