The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking

Findings of the Association for Computational Linguistics: ACL 2022, pages 3202 - 3213 May 22-27, 2022 c⃝2022 Association for Computational Linguistics The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking Yinghui Li1∗, Qingyu Zhou2∗, Yangning Li1, Zhongli Li2, Ruiyang Liu4, Rongyi Sun1, Zizhen Wang2, Chao Li2, Yunbo Cao2 and Hai-Tao Zheng1,3† 1Tsinghua Shenzhen International Graduate School, Tsinghua University 2Tencent Cloud Xiaowei, 3Peng Cheng Laboratory 4Department of Computer Science and Technology, Tsinghua University liyinghu20@mails.tsinghua.edu.cn, qingyuzhou@tencent.com Abstract Chinese Spell Checking (CSC) aims to de- tect and correct Chinese spelling errors, which are mainly caused by the phonological or vi- sual similarity. Recently, pre-trained language models (PLMs) promote the progress of CSC task. However, there exists a gap between the learned knowledge of PLMs and the goal of CSC task. PLMs focus on the semantics in text and tend to correct the erroneous char- acters to semantically proper or commonly used ones, but these aren’t the ground-truth corrections. To address this issue, we pro- pose an Error-driven COntrastive Probability Optimization (ECOPO) framework for CSC task. ECOPO refines the knowledge represen- tations of PLMs, and guides the model to avoid predicting these common characters through an error-driven way. Particularly, ECOPO is model-agnostic and it can be combined with existing CSC methods to achieve better per- formance. Extensive experiments and detailed analyses on SIGHAN datasets demonstrate that ECOPO is simple yet effective. 1 Introduction Chinese Spell Checking (CSC) aims to detect and correct spelling errors in Chinese texts (Wu et al., 2013a). It is a crucial research field for various NLP downstream applications, such as Optical Character Recognition (Afli et al., 2016), search query cor- rection (Gao et al., 2010) and essay scoring (Dong and Zhang, 2016). However, CSC is also very challenging because it mainly suffers from confus- ing characters, such as phonologically and visually similar characters (Liu et al., 2010; Zhang et al., 2020). As illustrated in Figure 1, “素(sù, plain)” and “诉(sù, sue)” are confusing characters for each other due to the shared pronunciation “sù”. ∗ indicates equal contribution. Work is done during Yinghui’s internship at Tencent Cloud Xiaowei. † Corresponding author: Hai-Tao Zheng. (E-mail: zheng.haitao@sz.tsinghua.edu.cn) Phono- logical 83% Input 希望您帮我素 (plain) 取公平。 s ù Correct 希望您帮我诉 (sue) 取公平。 s ù Candidate 1 希望您帮我争 (fight) 取公平。 zhēng Candidate 2 希望您帮我谋 (plan) 取公平。 móu Candidate 3 希望您帮我获 (acquire) 取公平。 h u ò Translation Hope you help me to sue and get justice. Visual 48% Input 我们为这个目标努力不解 (understand) 。 j i ě Correct 我们为这个目标努力不懈 (slack) 。 x i è Candidate 1 我们为这个目标努力不休 (rest) 。 x i ū Candidate 2 我们为这个目标努力不断 (break) 。 duàn Candidate 3 我们为这个目标努力不停 (stop) 。 tíng Translation We fight for this goal without slack. Figure 1: Examples of Chinese spelling errors. Pre- vious research (Liu et al., 2021) shows that 83% of errors belong to phonological error and 48% be- long to visual error. We give the characters with their pronunciation and translation. We mark the in- put confusing/golden/common candidate characters in red/blue/orange. The characters …

Content truncated to 500 words